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ABSTRACT 

During 1967 and 1968, TEMPO, a Research affiliate of 
The General Electric Company, evaluated federal assistance to local 
school districts to use for compensatory education authorized by 1965 
Elementary Secondary Education Act Title I. The specific research 
questions were straight-forward: (1) Has statistically significant 
enhancement of pupil performance resulted to date from Title I 
compensatory education programs? (2) What school, pupil and 
environmental characteristics are associated with enhanced pupil 
performance? The research design analyzed the extra gain, if any, in 
reading achievement of pupils who received the extra Title I 
services. The research tasks focused on the collection ol achievement 
test data given both before and after pupils were exposed to 
compensatory programs and the analysis of the association among these 
variables. A sample of 11 school districts yielded usable data for 
35,000 pupils. A second study allowed further refinement and analysis 
of the data from the sample schools to pursue answers to the same 
research questions as in the first study. Even though adjustments for 
possible negative trends in achievement were made and considerably 
more detailed cost data on compensatory education treatment were 
obtained, the previous negative answers to the research questions 
remained valid. (Author/JH) 
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Tne stu:iy v:as conducted In ;:ashinr;tcn, D.C. by General 
PJlect'-^ic - Tlh?0. /.rich is a par-* cf tne General P^lectric Cor- 
poraoicn. Crj^^nired :.n 113^ ^s a '^think tan>/' TZ::j: " conducts 
plannl/./, and evaluation st.^.dle^ /cr other or^anizat ic:.al ccxpo- 
nents oT G.T as v.ell as for state and federal governr.ental 
agencies. Its oreas of expertise in.clude strategic planning-, 
cost-i :nefit .-^naly-^is, technolcrical fcrecostin^, 
oval Ion. cf sc:;:a] cro.::rar.£. econcir.ic analysis/devclcp:r.ent . 
and irc;:':e:tal ir.pact studies. At tne ti>:.e of the study.. 
Tn::';rO nad about 300 professionals , i.ncludin^; Dcth its ;:ashingg 
ton office an:: its Headquarters in. Santa Barbara^ California. 



'^Alth:u^:;i a :..eir;cer of the professional staff of TE::F0 at 
the t::r.c of tf. is st\:dy. Dr. rCenn?!:: Gordon is presently with RKC 
Research Cor'rov.ticn In ?^v:thosd:^. Var^l^nd. v;hsre he las directed 
la/TC-': .nx^ic ^■'^lu'ition anl an::'\vlca3 studies in -ducaticn and 
other crj^s. \: 1* hns about 101 nr. i^essionals and carries out 
c^-) res.v-,-. rt\p^: suvh u.-.i^ 7r ;v:;?i:on. health, r -npov:er 
CNj troinr z-'.c:.'^ J.^ . ^ r?>.spc rcat : cn . urban developnont, 

o and oc5 t anal*, si., . f r. j:-'.rL^ara lordon v.an. previously a jonsultant 
i/^ 'mi ol^'.c/.tlov to flihPO, tai. :s Ti^rs'iXly nead of Gordon. Associates, 
vM ac If :ny .nc-.'..t xn.n.ltin;; firr: c;;s c is 1 ir : n - in educat'on pl^nninr 
^ and 0 valuat 1^.^^ . . 
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i;:troducti::; 



Durir.g l^/^l ar;d TK^PO , a research affiliate of the 

General Electric Vo.-Darr/j ccnducted ar. evaluation of a rr.aior 
federal edujaxicn ac3lsrar.ce prcra^ar^; under contract to the U.S. 
Office of Education. That proararri^ an innovative approac;. 
which provided federal grants to local school districts to use 
for ccrr.oencatory education, v:as fir^t authorized by litle I of 
the Eler.ontciry a:.d Secondary Zducaticn Act of I965. After Id 
months of rrcrrar. experience, there ..'ac a need to evaluate the 
effectiveness^of this acsistance approach v;ith the intent of 
serv: nr as an inout to the HEV; policy and legislative revision 
process. At the ca:r:e tirr.e, the govern:;.ent-v;ide application of 
the Planning. Pro^rranr.ing and hud-eting Systeir. ^(PP5S ) -..-as 
pressinr^ for'rr:ore analy^.ical evaluations of the outputs pro- 
duced by federal pro;:r8r:.s in direct relatior. to the inputs of 
those oror/rarr.s. Since this study purposely represented a nev; 
approach to help the evaluation (decision-rr.al-cing) process, there 
was no precedent to utilize for the research design. 

The specific research questions adopted for investigation 
were strai^jht-f orv;ard : 

1. Has statistically sigi.lficant enhancement of pupil 
perion:;ance resulted co dcxoe fi'oia TiLie I conioeii- 
satory education programs? 

2. VJhat school, pupil and environjnental characteristics 
are associated v;itn enhanced pupil performance (what 
are the distinguishing features of successful compen- 
satory education programs)? 

The research desi.^n utilized the extra gain, if any, in 
reading achievemeno (as^m.easureu on standardised tests) of groups 
of pupils v;ho received ti^e extra Title I services as the measure 
of effectiveness of those Title I projects. The research tasks 
focused on the on-site collection of existing achievem.ent test 
data given both tefore and after pupils were exposed to compen- 
satory programs and the analysis of the association among these 
variables. The croject included extensive field work in a 
sample of 11 school districts to obtain usable pre-and-post test 
data for 35, 00 J pupils as v;ell as extensive conversion of that 
test data'to facilitate comparison and aggregation across all 
the pupils. 

A negative answer vrxs obtained to the first quest icn--no 
clear overall enhancen.ent of pupil test performance was observed 
over v.hat woul i have otr.erwise been expected. This was true even 
thou..n m.os\: educators were saying this federal program was som.ething 



£^r^.at and fur/Jir.r for it should be expanded. Even thoae^i no 
gain v;a:: obGervec^ , the cecond que::tlcn v;as investigated since 
it is possible scrre subgroup or particular program trc^.trr.ent 
could have beer, haloed sisnif Icar.tly even ;f the overall aver- 
age was not. or course, separating cut such program efrccts, 
w}iich is difficult in ar.y analytical study, becomes a much 
more frustrating research task in the face of any clear overall 
trends. Thus, it may net be surprising that, even v;ith the 
apolication of several analytical techniques, no pattern of 
distinguishing features of successful compensatory education 
programs v;as identified. 

A second study was later funded by the Office of Education 
to allow further refinement and analysis of the data from the 
sample schools to pursue answers to i:he same research questions 
as in the first study. Even though adjustments for possible 
negative trends in achievement were made and considerably more 
detailed cost data on compensatory education treatm:ent were 
obtained, the previous negative answers to the research qu-^s- 
tions rem^ained valid. 



Pi^_RT I: rA::':o?.':v:;:. a::d ?:i:;t3 c? z:-:?hasis 



In 1965, the U.S. Congress initr'ated a nev; concept of 
broad federal assistance to help local school districts raise 
the perfor level 01 zr.e large ?;rcuc2 of students -..ho v/ere 

consistentiiy fallir.~, farther and farther behind their expected 
educational achieverr.ent levels. This program represented a new 
concept of airect federal invclverr.en- in the delivery of educa- 
tional services by local age..cies. 

This chaoter describes an evaluation carried out by Gen- 
eral B:iectric-TE:.:?0 cf the effectiveness of this new federal 
program after the first Ic r.onths of its operation. It is 
investigated if the program v.-as successful in "compensating for 
the educational deprivation of the target group of students. The 
contract from the Deoartm.ent of Healtr. , iducation, and '.-."elfare 
ent itled , A Surv ey and Prel iminary Cost - i:enef it Analysis in 
Elementa rv-Se ccn darv -du cat icn . resultcc m a com.prenensive 
TI77?;rTe3ort~^T7nrcrTF^iA^laole through the ERIC inforr.ation 
system). 3 A second contract follcv.ed imjnediately v.'hich further 
analv?ed the survev data and carried cut com.prenensive case 
studies of five districts receiving federal funds from this pro- 
gram. Because of its specific emphasis on mieasuring program 
effectiveness of a relatively :iew program, this evaluation 
effort oresented substantial difficulties in analytical method- 
ology and obtaining valid m.easurement data of student achievem.ent 
changes for a program already in operation. Nev; evaluation 
developm.ents v.ere needed since there were no previous studies 
similar in scope or purpose. 

The' assistance program being evaluated was established by 
Title I of the Elem.entary and Secondary Education Act of 19o5 
(ESEA) and is com.monlv known as "Title I." That legislative 
package (Public Law S9-IO), accom.panied by the highest appropria- 
tiops^cost for education in history for the nation's schools, 
was introduced at the "right" tim^e in terms of the clim.ate for 
passage of such a bill. 

The post-war period had several characteristics which en- 
couraged locpJL and state educational agencies to think in terms 
of federal sup:;crt for their school systems. First, th.ere was 
an unorecedentcd increase in students due to (1) the dramatic. 
Increase in tne birth rate and (2) the desire of m.any mere per- 
sons to rem:ain in school through high school graduation. Second, 
there was fm increasing deman.a for the gc}:oo1s to take on more 



recponniblllty fcr irr.prcvei Lr.toilc.tbal and techr.ioa] education 
for\-Au5er.t3: tr.-j .launoi. I:.:; of Scutr.^h ir.tensif iea tr : 3 derand. 
These factors pluc tr.e Supre:v3 Court decision::^ requiring de- 
se^re^-ation of sc:.oo2 3 ir. cornc; inat Icn v;ith an increased rr.oral 
concern fur the education of econorr.lcally and educationally 
disadvantaged children put nerfcofore unfelt derr.ands on tne schools 
of the Unired States. Previous to this period, "Arr.erican ^iduca- 
tion had been a highly diffused enterprise over most of the 
course of its history. The role of the Federal goveri.rr.ent , 
particularly, had been r.ir.irr.al and any suggested increase in 
that role had been generally feared and fought J'^ 

In the period of the 19503 and 1960s the der.ands on the 
financial resources of local and state governments in the field 
of education began to overcome the fear of federal money being 
added to school budgets. The largest organization representing 
professional educators, the llational Education Association, was 
at the forefront of urging federal aid to education, r^ven 
though this Association had been on record as favoring federal 
aid since //orld War I. its lobbying efforts drastically increased 
following /:orld V.ar II • These realities and efforts coupled 
with a m.ore responsive political climate in 'Washington resulted 
in federal aid legislation in (1) vocational education, 7 (2) criti- 
cal, defense related areas, ^ (3) educational research, 9 and -j^q 
(4) aid to the educationally disadvantaged and handicaci^ed child. 

ESEA was proposed at a politically feasible tim.e-- shortly 
after President Johnson cam.e to office and developed his //ar on 
Poverty. It v;as in this context that the ESEA was signed into 
law on April 11, I965. O^st 39 days after it v;as introduced In 
the Congress. That short period is highly unusual for such 
major legislat ion. 

Title I was by far the largest financial segm^ent of the 
ESEA. The funds appropriated for this title have varied from 
950 million dollars in^ Fiscal Year 1966 to alm.ost 2 billion 
during Fiscal Years 1972 and 1973- Title I had (and still has) 
as its puroose the provision of federal financial support to 
Local Education Agencies (LEA) drawing their students from con- 
centrations of low-incom.e fair.llle^. The Act assumied that a high 
correlation exists betv;een educational deprivation and low- 
incom^e. In comiection with ESEA, Title I, Cox^.gress m.ade this 
declaration of policy: 

In recognition of the special educational needs of 
children of lov;-inco:r.e familif-'S ana the impact that 
concen.tratlor.s of low- income families have on ti.e 
ability of local education agencies to support 
adequate educational program^s , the Congress hereby 
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declare^ it to be tr.e polijy of tr,o United States 
to provide fir.ar.clal acclctar.ce to lojal edu:;aLior.a.l 
agenciej ser/in^ areas v;ltr. conceit rat ions oi child- 
ren fron lo-.>in2orr.e farr.iliec to expand and ir.prove 
their educational prcrra!:r3 oy various rr.eans v;hich 
contribute particularly to rr.eetir.r; the educational 
needs of educationally deprived children. H 

ESEA v;as enacted so ouicivly that r:.any LSAs were caught 
totally off ruard. I^ost of the initial Z3ZA Title I prograrr:s 
did not result frcrr. an analysis cf the needs or accurate, objec- 
tive definition of the educationally deprived at the Lz,A level. 
ESELA Title I n.onies v;ere received by all states and three-fourths 
of the LEAs. The poverty indicators v;ere specified in the Act 
as : 

Children ar.ed 5 throu^-h 17 in families with an 
annual ir.coir.e below $2,C0O<, according to the 
i960 census, and 

Children in the sarne are group from families 
with an annual incon.e above o2,0D0 who receive 
Aid to Farfiilies with Dependent Children (A^-DC). 

The total of the children in the eligible categories --'as multi- 
plied by one-half of the State per pupil expenditure to compute 
allocations. The state educatic;. agency administers the pro- 
gram and suballocates funds to projects proposed by eligible 
local education agencies. Although projects are to be submitted 
for aooroval irom^the local district to the state agency, in 
rcalitv few orcposals are rejected. In the first few years of 
Title I, more than one-third of the total expenditures were 
uuilized for reading projects. Among the other projects often 
funded were those in such areas as library, guidance^ and counsel- 
ing^ and instructional programs such as mathemat ics 

Each local district is supposed to utilize program funds 
only in areas with high concentrations of children from low- 
income families and in those areas, funas are earmarked to m.eet 
the special educational needs of the educationally ueprived. In 
the beginning years of ESEA. m.uc;^. confusion resulted from attempts 
to pinpoinx eligible recipients of this aid. 

The basic purpose of com.pensator,/ ducation ^(provided by 
Title I hSEA or other rrcrrar: ) w-ts 10 :lo3e the *'gap ' in the 
educational achievem.ent' of ^ certa i!* stud:>^.r ..-roups, a performance 
gap which gets 'wider as tim.e progresses, If conditions dcn^t 
change. The education gao botveen pupils attending schools 
eligible for Title I assistance and non-Title I schools is 



si^-riif i::ar.t and critical, rcr cxarr.ple. the statictical aistri- 
butler of ..ect scores for reaulr.^ (or ar.y other parti-jular 
student snilj ) Ir. a certair. rr'^d-^ ir. a Title I school typically 
stands v;ell to the left of tho correspor.ding districut ior. ta>en 
from the overall national picture^, as indicated by 7iqure 1, 
The arr.ount of difference varies of course, but. :n general, 
fourth grade pupils in Title I schools lag about one year 
behind national^ achieverr.ent levels. A longer-terir. consequence 
of this problerr. is illustrated by Figure 2, The norm-achieving 
pupil acnieves at the rate of 1.0 grade units of academic pro^ 
gress for each school year. Tne typical disadvantaged p^jpil 
progresses at a rate equal to only c5-75 percent of the norm. 
Thus, after 12 years, a disadvantaged pupil can be as mu'jh as 
four years behind, if he is still in school. Cne of the goals 
of compensatory education (C^) is to improve the achievement 
rate of these disadvantaged pupils and thus lessen the gap 
betv;een them and the national norm. As noted belov.% the nature 
of these educational needs underlying com.pensatory education 
had significant effects on the evaluation design and process 
of tnis study. 

Publication of the Cole:;.an report, gquality of Educational 
Oooortuni ty , 3 ^as led to t\:j ccmjron misconceptions amcv-Z eau- 
cators and the general public regarding the relationship of the 
report to Title I ES2A. The first of tnese is t^e acsum.pt: on 
that the generally negative results of the Colem.an Report is 
proof that com.pensatory education cannot be expected to v:ork 
and the second' is that SSEA legislation actually v;as the out- 
come of the Colem.an study. I.'ote that these are divergent view- 
points. To dispell misconcertions about the second item.- it 
is only necessary to note that the PL B^-IO ESEA ?ill was 
introduced into the U.S. Congress in January I965 and was passed 
on Aoril 11^ 1^-65. The Colem.o-n Report was a study done in 
response .to the Civil Rights Act of l^c^l^ and was not published 
until July I966. 

On the first point, although the Colem.an Report did indi- 
cate that, using existing educational practices, there was little 
evidence to indicate that additional spendijng is an important 
factor in im.proved school achievement, it did not evaluate situa- 
tions in wh.l.ch Title I money '..'as being utilized. 

The Coleman study also pointed out a reality of educational 
evaluation brought hom.e by oiv: present study--the difficulty^ of 
m.-asurom.ent . in comment inr c. the l^icr: of inform.aticn 1:^. LLAs 
on whic:-. to evaluate or gaure needs, t::e Colem:an Report: Indicated 
that ^'one of the real handicaps to an effective assessment of 
equality of education for children of minority groups (or any 
children.) is the fact that few comj-unitios have given systematic 
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Figure 1. Typical education gap found 
between Title 1 fourth-grade 
classes and the national norm. 
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to:3ting and fev/er nave e/aluatod zY.e academic perf ori: ^•.\':te and 
and attitudes of these children icv:ard edu::at ion. ''1: 



POLICY rACKa?.DU::D 

At the tlrr.e this study v:as initiated by DHZ'.-; , several 
important policy questioris v;ere tariing shape. The arialytical 
exarp.ination this project provided ;:ao considered cne important 
contribution to ansv.ering^ those pclivy questions. 'Ine overall 
issue v;as : "::ow that this innovative prograrr: has been operating 
for over a year, v;hat have been iis effects cn students, in 
relation to the inputs. '* Criticisr. had already been voiced 
frorr. both inside and outside tnj educational coirr.i:: J ty ) that 
itle I v/as not achieving its co.lecoives . v;as prirriarily serving 
as general aid to local education froir: the federal sector, and 
was not reaching the intended target group. 

V/hile the regular procedures of the states, the General 
Accounting Office, and the Office of Education v;ere -onitoring 
hov/ Title^I funds v;ere allocated and spent, the question of 
effectiveness and inxaots on students required special rr.easure- 
ment of achievement and analysis of inputs to outputs. Up to 
that tirr.e the Office of Zducation had not addressed chis sort 
or question and was not in a pes it ion to obtain or cx:.ci.iy^e Ihctl 
type or student achieverr.ent and pro£:rar. cost data. r;c-.:ever, 
increasing concern v;as being voiced by legislators and other 
policy groups about v;hat they vere getting for their ::.oney , 
and hov; they could change the prcgram or projects to increase 
effectiveness . 

There were three n:ain ways that DK3::J and OE could take 
or recoimr.end) action in the policy area to n:ove events tov/ard 
esired ends. In fact, these are areas where the agencies are 
obligated to periodically review and reco!r.r:end the necessary 
actions, and they were seeking helpful inputs to that process. -^^ 

(1) Allocation of Future F unds. Title I funds could 
be allocated u^i'i-ereniiy next year by CE to th.e 
states and LEAs . Eotji the overall level and the 

•relative distribution of funds could be r.odified. 
although the ESEA legislation contained certain 
allocation constraints which had to be followed. 

(2) Revision of Tiile 1 Gui::elh'.es . Since CE prepared 
the detailea ?rcce::urcc a-. a publi^-hed ruide3jn.es 
for implementation of Title I, needed modifications 
could be made^ These guidelines govern the pro- 
cedures by which LEAs apply for Title I funds, how 
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eligibility is defined, ;;r.at Icital activities can 
be supported by Title I funds, and what evalua- 
tion actions are required cf LlIAs. 

(3) Recorjr.endati on of !i3v: Lerislation . Eased on the 
exoerien'j'j c:" t:'.e I'lrst year cr tv;o v/ith Title I, 
DHS'.i could recorr.-'.cnd rr.odif icatlcns to 53ZA it 
believed ne:;essary or desirable. Since hSr-A 
had been a nev; approach^ rr.any Congressir.en were 
particularly receptive to assessing its early 
impact and rr.a^ing needed changes (several pack- 
ages of ar.endrr.ent? \;ere actually enacted over 
a several year period). 

The existence of these policy needs coincided with the 
erierging desire co try better evaluation approaches focusing on 
the effects or outputs produced by specific federal programs. 
The TE:^:P0 evaluation contract being described v;as one result of 
the combination of these forces. 



TH5 mOCURE:.S::T PROCZSS 

The procurement process for this study did not follov: the 



ment process by TSVPO and the changing interests (and people) 
within 

It is important to understand the situation within HEW 
and USOZ at the time this study was initiated. Historically, 
neither evaluation nor its use in the resource allocation pro- 
cess had been very frequent or intense. Curing 19c5 a fresh 
attempt to improve decision-making was occurring in several 
civilian departr.ent s , including VZY: . The stimulus v:as provided 
by the apparent success in bringing structure and quantitative 
analysis to the m.ilitary planning process in the Departm.ent of 
Defense (DOD) during the early IS'COs . Evaluation of program out-- 
comes and effectiveness was a m.aior cornerstone of the crocess 
of program budgeting and systematic allocation of resources across 
program.s. Under the banner of a Planning, Prograrjning and Budget- 
ing System (PPBS)-, all federal governm.ent agencies were instructed 
by executive order of President Johnson in August 19^5 to adopt 
sim.ilar principles and procedures as had been developed in the 
Defense ^.partir.ent . (I^any cf the directors and key staff of the 
planning or analysis groups within HIV.' and USOE at the time of 
this study were forr.er m.emcers of the Defense Department or its 
contractors . ) 
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Durinr that car.o tirr.e nericu, TZXPO had similar ''bcec- 
xives. Sev-ira} ai:-:?0 staf:' v,'cr>ed •..-ith DC." ar.i also 

were ir.tere::LC'< ir:' aDD.yi:.;: tne ca:;.e apprcach-as to pr./cier.s of 
the civil: ar. a:-or.oie3 . TLhPO Vias :r. the business of carrying 
ont such research ar.d Dlanr.ir.g sT.udles and additional -cntract 
activicv v.-ould aid in it-; grcv.-th. Cne of its organizational 
ob.jectives -..as to diversify frcr: its past depender.co cn contract 
work in defense strategic planning and military anaiy.-is. 

U3CE and TEMPO'S interest coincided in Decer.r,--.r 1965 v;hen 
KS'.-; solicited letters of interest from ?. i ^ organir.at ions 
via a: announcement in the "Cor-.merce Daily :;ulletin ' (December 
2':), 1963) to carry out the follo;ing actions: 

(1) Provide a f ramev.'ork for system.at ically ordering, 
relating anu integrating educational goals, pro- 
grams, and program, elements. 

(2) Permit systematic identlf icat iO!i and analysis of 
alternative educational goals and programs, and 
their short and long term, requirements and effects 
(in terras of cost-benefit factors and economic 
criteria appropriate to policy decision). 

(3) Provide a basis for systematically identifying 
inform.ation requirements for educational planning, 
programmiing, and evaluation activities (of USOE). 

(4) Permit continuing evaluations of progress in 
meeting approved goals. 

As these points shov;. the reouest v.as structured in the frame- 
worl-: of Dror.ram budgeting and providing help to the policy 
miaker in'oi^der ing priorities, allocating resources, and evaluating 
imipacts . 

T2'-'P0 submitted an exoression of interest and a brief 
descrioticn of its caoabi lilies at that tim.e. Inform.al discus- 
sion? ".ere periodically held v/ith iiEVJ-USOE -off icials over the 
next T-ev; mo'-^ths v;henever the opportmiity presented itself. No 
studies resulted from this activity in the si-.ort run. In March 
of l<c7. v.'ith the encoura-rement of H::.V,'-U30E staf f , ^/.-XFO sub- 
mitted ai' unsolicited oroocsal to USOE to conduct '^i Survey 
and p>-.oT =r-i -'.-.^v Cost-Tonefit Analvsis in iClementary ard Second- 
ary Ed-ocaticn"' (TEVPO lie. l-cy'-.Z). Ihe proposal v.-af prepared 
comoletelv bv th.e senior staff of TEMPO'S V.-ashington office, 
primarilv' based on their cxpcriei-.ce in assistin- ^^.overnment 
decision-ma.kcrs in oror.ram bud-ret ing, ratner than experience m 
the field of local education, li-.e unsolicited propo.-^al outlined 
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study objectives and pro.ject activities but the developrr-ent of 
ir.ost of the analyti::;al approach a!.d recearch design .vas deferred 
until the actual study. It v;as considered an exploratory study ^ 
i.e., a test of v;hat a systerr.atic analysis of this type v;ould 
produce \.'hen applied to the field of cduc-"'-' * - Even sc. it 
v;as assumed that the results of this stu . help ••;ith 

!T;aj*or funding and prosra^i decisions that n^j. to be ir.ade in the 
Fall and V;inter of I9S7. 

In April, 1967, Contract HZW-CS-67-p5 v;as av;arded to 
TEMPO to perform this study. The prefect office for technical 
guidance and monitoring v;as specified as the Program Analysis 
Group at the HEV; level rather than within USOS itself .1? A 
letter to the chief of the HIVJ Procur er.enr Section on April 
20^ 1967, to justify the av;ard of the research grant to TEMPO 
stated that ''USOE had decided to transfer funds to the Office 
of the Secretary, DHE;J" for the proposed project. The cent act 
v/as for approxima-cely $175,000 (on a cost-plus-fixed-fee basis). 
The second contract that followed in 1963 was for apprcxim.ately 
$100,000. 



D. WORKIIIG RSLATIOIISHIPS 

v/ork:ing relationships between the contractor and Ihe client 
i^ere fairly effective • The main contact \;as with the HEV7 Pro- 
gram Analysis Office where trie contract was issued and where the 
contract technical monitor was located. The monitor maintained 
relatively close monitoring and coordination of the work pro- 
gress. TEi-'PO sought to maintain this involvemient throughout 
the study, partly bec^-use of the evolutionary nature of the 
research task. For exanple, during alm.ost all field visits to 
school districts for data collection, TE::P0 sought and obtained 
joint participation from either HSV7 or CE personnel. In m^ost 
cases this was extremiely beneficial since "Che otherwise limited 
opportunities for travel to the field m.ad-^ it difficult for HEVJ 
and OE staffs to have a good understandir.g of the Title I acti- 
vities being carried out at the local level. The location of 
all TEMPO project staff in VJashington, L. C,^ made it m.uch easier 
to mieet often with various mionitoring staff mem.bers and ^o keep 
them involved in both the deve3opm.ent of the technical approach 
and the data collection problems. 

Although the prim.ary person responsible for monitoring this 
contract in HE"/: chanf^ed during the life of the complete project, 
no difficulties resulted frcm^this actic!i. VPneri the director of 
the Program. Analysis Group at HEV/ left to becom.e presideiit of 
the newly formed Urban Insitu-e,, his deputy took over his position 
and also continued to m.aiutain xecnnical coordination witn this 
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contract. At a later poi;.t anctr.er ::-jr.ior staff member assumed 
major re: pcnsibility for mor.i^or :r.:V tr.o fir.al phases of z'r.e 
initial co:. tract arid cor.tinued thrcurh the extensive investi- 
gation of five districts during the second contract. 

Tno Title I Program Office v;ithin OE had involver.ent in 
the field sta>5e v/ith the project sir.ce its staff members parti-- 
cipated ir. the joint field data collection team.s. The central 
research and evaluation group at the Office of Education level 
v;as essentially uninvolved also although it is assumed trey 
kept informed of the research progress at tiie same tim.e they 
v;ere pursuit.g their ov/n research objectives. vn:ien the project 
was comnl'^ted;, briefings on the results v;ere given to repre- 
sentatives of these and other observed on the part of tre staff 
of the OE Program Office admir-ister j Title I. This group 
establisned guidelines and monitorea ^r.e Title I progra:..s at 
the state a>id local levels^ and did not appear to feel threatened 
by this outside evaluation of the program they were administering. 

The other m.ajor group v;hose cooperation was needed v/as the 
officials of the school districts being examined. With few 
exceptions, these relationships worked out well and adequate 
cooperation was received during tl^e data collection process. 
This m*ay have occurred for three m.ajor reasons: 

(1) advance concern for this problem was evidenced by 
TEMPO and OE, including particular care v/ith 
advance preparations for field visits^ e-g-- 
courtesy calls to superintendents and state educa- 
tional agencies. 

(2) almiOSt all data collection was don on-site by 
TEMPO and OE staffs thus m.inim.izing the amount 
of extra work and confusion on the part of local 
'staffs and 

(3) this v;as one of the first federal evaluations of 
ESE/i (LEA administrators iray have felt different 
later v;hen several evaluations were underway). 

Relationships were nearly alv.ays very cordial between the on-site 
team^ and th^ LEA staffs once the personal, face-to-face stage 
was reaciiod. In som.e cases ^ the local staff went considerably 
out of tl eir way to provide supplemental data requested by TEMPO 
at a later point of tire. Kosiility or lack of cooperal: on was 
encountered in only two or tnree scnool districts of t]:c original 
sample. Inese districts ^ere dropped from the sample at an early 
point since it v;as apparent the necessary data would not likely 
be obtainable within the sl^crl time available. In m.any cases ^ 
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midclo-level Iccc^l Gtaff v;ere extre:..ely helpful and receptive 
sir:c:o they v;ere ent-uc iact i:j at the prospeet of havir.g outside 
professional:?: intererjted in their data and their problems. 

Some difficulties ;/ere encouj;tcred v;ith the initial allo- 
cation of both time and money for this study. For one things 
a very optimistic schedule v;as initially established (the final 
report \::thin six months). V/ith the approval of HE/.'^ the final 
report v;a3 actually submitted in March of 1S-6& (a total dura-- 
tion of 11 m.onths). The major contributing factor v/as the 
considerable effort required by the Joint team.s in obtaining 
the necessary achievem.ent data from local districts. Also 
contributing to the longer schedule v:as the large am.ount of 
data processing necessary to convert the individual student 
test scores to a ccmjr.on standard a:.d to statistically analyze 
the larre amounts of data. Even after the fixlal report '..'as 
cor.pletod for the original study, there were several additional 
questions and data refinements that v-ere mutually considered 
v/orth investigating. A second phase study was designed and 
funded by OE to do further in-depth work with five districts 
of the original samiple . 

Due prim.arily to the expanded schedule, the original fund- 
ing leveJ :;as extreir. ;ly tight and tnis was a continuing constraint 
on fbp c;pon9 of TTn'-iPJi^s activities. The second phase '.'as 
comipleted within the funding levels and tim.e established for it* 
In interpreting the sufficiency of both timie and money for this 
study^ it should be remem.bered that there was considerable un- 
certainty about the specific technical approach when the original 
estim.ates were ir.ade. That, of course, is \-:hy it was thought of 
as a research study and w^hy the funding level was established 
und'er a cost reimbursemicnt arrangemient . There was a considerable 
amiount of exploratory work involved and OE and TEMPO mutually 
agreed to several revisions to the technical approach during the 
life of the project. In retrospect, it is probably not reason- 
able to expect that the timie and m.cney required could have been 
accurately estim.ated prior to the study since the analytical 
approach had not beeii used before on that scale and because the 
staff involved (in both HK'/ and TEMPO ) were new to educational 
evaluation. 

UNIQUE ASPECTS OF THE STUDY 

There are several poi!:its about this c^udy that are worth 
special m.enticn. To begin with. a. group of unique characteristics 
of the technical approach dif f ererit iate it from many otner studies. 
At th.e tim.e this study was initiated, it was not comjnon in the 
field of education to specifically investigate effectives. ess of 
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educatior. (ti.at ic. to 2o:r.par'"; the r.orforrr.ance and value of 
outputs to trie ::pe;lfi'j ir.puus of ^:^e process). Most r.aoional 
educational or psyohoicrical rrcudies previous to this t Irr.e 
v;ere descriptive in nature ar.d focused on the educational pro- 
cess itself raxhor than the output product of that process. 
Title I ESEA *.;as a nev; approach, an att3:r,pt to compensate for 
educational disadvantaTerr.ent via added federal financial 
assistance. a result of these factors^ TEMPO had no national 

educational precedent to follcv: in designing the technical 
approach and evaluating this educational program. But. as pre- 
viously described^ it was also true that initiation of this 
particular study and its a'.;ard to an analytical study group like 
TEMPO v:as a direct reflection of the desire of several HIV; offi- 
cials to bring nev; objectivity to ohe evaluation task. They 
v;anted a cor.prehensive analytical lock at the problems involved 
to help educational planners at the federal level rr.ake decisions^ 
and they focused on coir.pensat ory educational for the disadvan- 
taged as the irni.tial application area. Reflecting t'ne beliefs 
of severa] key r^LVr program analysis staff members, the approaches 
successfully used for planning, prcgrarrjning^ and budgeting in 
the Defense Departm.ent v;ere applied zo seek understanding of the 
impacts of large-scale federal program.s in education. 

Another unique aspect of ti'.e technical approach centered 
around the need for lar^e scale ^oi..parati ve acliie vcincjnt Leboin^ 
of the pupils. Because theie v:as neither time nor ir:oney to 
adm.inister student achievem.ent tests ^ THMPO had to depend on 
tests already given by the school districts. This^ of course ^ 
meant that cooperation of the LSAs ;;as needed in identifying 
and m.aking available pre- and post-test data on their Title I 
pupils, it also precluded use of a random: sample. Moreover^ 
it v:as necessary to m.ake personal field visits to the sam.ple 
districts to obtain student test scores and other data. The 
necessity for using acriievem:ent test results whi^^h v/ere avail- 
able created great difficulty in m.aking com.parisons across 
districts. ^ven though TH:mPO used extensive statistical and 
data processing procedures in standardi;.ing test data to enable 
cross^ com.pariscns • this did net elim.inate the lack of com.parabi- 
lity as a^ source of potential error. The tests v:ere just not 
designed or norm.ed by their puolishers to be comparable to each 
other. This difficulty of obtaining achievement data on indi- 
vidual studpni-s over successive tir.e periods required comparing 
the perfcrr.ance of a whole grade In a school (fixed grade approach) 
rather than the perform.ance o-" individual students over tim.e 
(the lor.rirudir.ai approach ). -"1-5 A.l though the longitudinal approach 
would have been preferable since it eliminates several outside 
sources of variation, it would have required the retrospective 
tracing of individual students from grade to grade (and perhaps 
their migration to other schools). It also would have required 
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school districts to provide test iata to TEMPO by stude'.t r.arrie 
rather than the statistical S'v.n.:.ary cy ^jcb.^^ v.t.ich rr.ost -cnocl 
districts had available (scrr.e cited confidential cor.cei-^s for 
not providing student nar.es). Up to that point in tin.-:-, fev/ 
if any evaluation studies had been dene that required s'jch 
large scale and coir.parat ive use of tesx data. Student d.chieve- 
ment tests v.-ere developed and norr;-.aliy used only for individual 
diagnostic purposes or ccrr.parison of group progress v.'ithin the 
same school or district. 20 

The seccrd ra.jor ir^^up of points to emphasize about the 
TEMPO study concerns the logistics r-.anagerr.ent and v.'ork :"orce. 
The study v;a3 ccr.pletely carried out by staff rr.err/cers of TEMPO, 
all located in the VJashing-^on office. There v;as no sub-contracting 
other organizations. The staff v.-as cor.pcsed of professionals 
in such analytical areas as econcT.ics, r.atherr.at ics . staListics, 
and ooerations research, but there- v.ere no professional educa- 
tors on the full tinr.e staff (five senior staff held Ph.D.s). 
Two Dsyoholccists v;ere involved full tixe during certain phases 
of the study.' Experts in such areas as psychological testing 
were used as consultants for short time periods. During the 
conduct of the study, this '-.cmpos iticn of the project staff was 
considered sat isf artory by botn TuMPO and :-:'£>:. In retrospect, 
some additional involvement of specialists m:ay have been bene- 
ficial, particularly during the conversion and interpretation 
of achievement test data. Of course, the composition of tne 
project team at TLMFO reflected fairly directly the philosophy 
of HLV/ in initiating this study. 

The logistics and field v.ork necessitated by the require- 
ment ^or field collection of large a:.-.ounts of data had a great 
effect on several tastes of the study. The need to have coopera- 
tive LEAs that would m.a.:e the necessary data available had an 
effect on. the sam.pl=> selection process and eventually required 
a reduction in the desired sample si2e (and later the dropping 
of three districts -jbic'- did not make the righ.t test data avail- 
able in tiriS). TEMPO also found that it could not depend on 
school district budget and planTlng informiation to determine the 
exposure of students to cor.pensatcry education. Field observa- 
tion and study found that data concerning actual teacher assign- 
ments and .rem.edial activities were needed, particularly regarding 
what Title I orogram activities v.-ere actually carried out (i.e., 
it was not sufficient to depend on the planned activities as 
described in the district's applications for Title I funds. 

Even thoush the extensi\e field data collection effort 
created the aifficulties descrioed, it had several beneficial 
aspects. Undoubtedly, better data resulted from the personal 
on-site efforts of TEMPO staff. The specific data needs could 
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be descrioed to LZA staff bettor ar.d questions ansv;ered cn the 
suet. The 1 a::e-to-f ace relat lc:.:ihlc tr.vs established al^.ost 
always produced better cooperation and r.appier school district 
staff. In n.any cases, TZXFO staff did the necessary clerical 
\']or> to extract needed data e.z the district offices. ('^he 
alternative would have been to have Lr.A staff fill out a data 
collection forr. and return it by rr.ail) . in some cases. T.iV?0 
staff collected or est irr.a-ced original resource data that v:ould 
not have otherv;Lse been available to the district. In addition, 
the on-the--s?ct observations of operating Title i activities 
irr.proved the*^ interpretation of the results of data analysis. 
The personal on-site collection of data also increased the 
response rate considerably. 



F. KEY fii:di::gs 

Tht5 key findings of the TEMPO study are surrr.arized here. 
In a descriptive sense ^ TEMPO'S on-site exairiinat ion and sub- 
sequent analysis of Title I activities found: 

(1) a very wide variety of coir.p-.nsatory education 
activities being conducted with Title I funds;' 

(2) compensatory education activities spread thinly 
over a large number of students; 

(3) considerable confusion and vagueness am.ong school 
officials on the objectives and allowed activities 
of Title I; and 

(4) considerable difficulties and dilution of impact 
due to bad timing, e.g.. Title l funds were re- 
.ceived very late and future r'unding was uncertain 

(many school planni!ig decisions were already m.ade 
before Title I resources were defined or available). 

The results of TEMPO'S analytical approach and data analysis 
were as follows: 

(1) No statistically s'ignificant enhancement in student 
achievem.ent was^ found over the first li years of 
Title I activities based on pre- and post-tests for 
about 25,000 students (in fact, a slight decline 
\vas observed in mea:: perf or:;.c.nce ) . 

(2) Some separate favorable situ:itions were encountered 
and available longitudinal data in one school dis- 
trict revealed significant 0!.i:ancement . 
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(3) Separate analysis of 7j.riar.ce calculations 
utilizir.g se /eral chaz^acterlst Ics of the LEA, 
students, and Title I activities (including 
per-pupii expenditures) revealed no consistent 
pattern of associaticr.s that v;ould explain the 
lack of overall enJian cement . 

(4) Subsequent adjustment of performance data for 
past trends of the schools and added analysis 
of applied Title I resources at the grade 
level during the second contract did not modi-- 
by these findings appreciably. 

In sumir.ary, TEMPO could not shov.: any effectiveness (impact) 
on student achievement levels from the first l-> years of Title 
I program.s, even though local educators said the additional 
money v:a3 useful and larger amounts v/ere needed. 



G. IMPACT OF THE STUDY 



It is difficult to assess 
lack of positive results about T 
policy makers. Som.e people v;ere 

educators doubted the results, 
all the extra effort and money o 
dents in scm.e v/ay. wliile TEMPO* 
to existing dioagreements on the 
assistance initiated by Title 1^ 
issue . 



the specific impacts TEMPO ^s 
itle I programs had on federal 
already critical of Title I 
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They basically had faith that 
f Title I had to help the stu- 
s completed study added fuel 
value of federal education 
it clearly did not settle the 



Possibly ;> Title I v/as effective and the study just didn't 



pick that UP--TSMPO readily admitted 
There v;ere several very good reasons 
positive achievem.ent gains could be 
av;ay by som.eone inclined to continue 
education. To nam.e a few: 



this possibility existed 
v,hy the lack of strong 
rationalized or explained 
suppcrting compensatory 



(1) The sample utilized by TEMPO could be construed 
as non-representative because of the needs for 
school district cooperation and extensive data 
collection . 



(2) Title I represented a very ne' 
it v;as too soon to expect any 
achievement gains to show up. 



; program and concept; 
significant student 
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(3) Standarciized achievei.'.^'-it t-3st3 rr.ay not be aaec/j.ate 
measurec cf studer.t prcvrcss . particularly :"cr tr.e 
highly disadvantaged students '.;hieh v;ere the target 
of Title J prograrr.s. 

(4) The inability to separately identify and measure 
progress of individual studsnT:s receiving signi- 
ficant arr.ounts or' ccrr.pensatory education help 
(thus requiring consideration of that grade as a 
v/hole)^ ir.ay have led to n.asking of achiever.ent 
gains. 

Given the findings cf TZ.vpO ' s study, what impact cn the 
policy-rr.ak:er should the evalua-cor expect? Surely it * s not 
reasonable to expect hin co coi.-.pleteiy drop a new federal assist- 
ance concept that is just getting cff tne ground. Neither the 
quantitative nor cualitati.'e evidence of this exploratory study 
v;as strong enough or clear-cut enough to justify that action or 
to overcome the strong suppcrt from the educational corrjr.uriity 
to sustain widespread financial aid to Title 1. 

But v;hat the negative results of the study should h.tve 
done v;as to raise caution and sound v;arnings that maybe this 
nev; program v;as not fulfilling its m.ajor objective of imiproving 
the pnuo^? t1 r.np 1 progress rate of disadvantaged students above 
established levels. Certainly these resvlts should have precipi- 
tated the further testing of the hypotxicses im^plied by the TEMPO 
study. Rer.er:.ber. at the tlm.e the Tl'lPO study v;as cor.pleted. its 
lack of affirmative support of the effectiveness of com.pensatory 
education v/as counter to the prevaili^',g opinion in the educational 
cormmunlty. Therefore^, it is not reasonable to insist (or to 
expect) a gcvernm.ent policy-m.aker to suddenly desert the concepts 
and procedures of Title 1. especially v;h.en no alternative federal 
assistance for the disadvantaged v:as a'/ailable (unless the recoin- 
m-endaticn of scm.e policy-makers v;as accepted that Title I be 
used as a Vehicle foi^^ the federal gcvern.m.ent to provide general 
educational aid to local school districts) . Kov;ever^, it is 
reasonable to expect federa] policy-m.arcers to refine federal 
guidelines and procedures for TitHe I (within the fram:ev;ork: of 
existing legislation/objectives) in an attem^pt to increase the 
program* s effectiveness. 

Several of these expected re.sults did occur. There v:as no 
tendency to curtail Title I fundiiig in the afterm.ath of tne nega- 
tive results cf t::e ii'itiai study. Cf course, the second full 
year of Title I activities vas already iir.derv;ay by the time 
TEMPO * s final report v,as published. Even for the second year 
(and even ir.ore so for the third year of Title I), the application 
procedures an.d federal guidelines for Title I v;ere refined in 
severa.1 ways. Efforts were un.dertaicen to get funding out earlier 
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and to ore vide ad\ar.ce ir.dicatio:i cf ar.t i:jipated fur.air.g. At 
one ooint durinr later y-?ar3 of Title 1, a two«year funding 
autr orization v;as obtained fro:.-: Ccnrreos (an unusual occurrence 
in itself). In lie;ht of observations on tne, lov; concentration 
of resources per pupil (both frorr. fr.e 'I.-^PO study and other 
observations)'^ efforts v;ere n.ade to ensure the concentration of 
available Title 7. resources on a srr.allar number of disadvantaged 
students. For exarr.ple , California recc:rnlzed^the iir.portance of 
concentration of expenditures in licY-zz/^- it's annual report 
indicated "the eva]uative results zv^::^st for optir.vj.m effective- 
ness the average student expenditure :r.ust be--$300 over and 
above the regular school progrann This recorrjnendation became 
part of the Title I regulations for California in IS-cQ-TC, 
tneref ore . reouiring further concentration of Title 1 r.cnies 
for the disadvantaged. 22 hZW also tigntened and eni'orced 
regulations so as to ensure that Title I resources v;ent only to 
qualified schools and that compensatory education resources v;ere 
not used as a substitute for regular expenditures by the school 
district . 

Subsequent to the TEMPO study, the U. S. Office of Educa- 
tion also moved to collect r.oro evidence and carry out more 
evaluation on tne effectiveness cf com.pensatory education. Large 
scaJe evaluations utilising reading achievement test scores v;ere 
conducted for the 1Q&7-65 and i pr,prr,G sn"r ooi years. They u?^ed 
data collected from a large r.ail survey of a random, sample of 
schools from the nev/ly initiated Belmont joint federa 1- state 
educatior.al inform.ation system. The 10c7-c8 evaluation effort^^ 
was based on the selection of a nationally representative sam.ple 
of pupils in grades 2, and 6. Kov;ever, the report indicated 
that scores on non-paraliel pre-program and post-program tests 
were reported for only 25,103 pupils, and analyzable scores on 
parallel pre- and post-tests v;ere reported for 11 Ali'O pupils, 
aporcximiately 9 percent of the returned pupil questionnaires. 
Despite the sparseness and non-representativeness of th.e achieve- 
ment data, some cor.clusions regarding the compensatory reading 
programs v:ere given: 2^ 

(1) Schools selected those pupils with low scores on 
reading to participate in comrpensatory reading 
program.s . 

(2) VJhere data collection allov;ed comparisons between 
students who participated in specific Title I 
program.s with tr.ose who did not participate, the 
conc3usions were: 

a. For both groups, their progress in reading 
skills wa3 3lmii:^r to thc:r past progress. 
Therefore, the T: tie I students never 
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progre.s^jed fa^.t enough to catch up to their 
non-Titlo 1 classmates. 

b. Reading achievem.eni. for both Titlo I program 
students and for non-participating students 
grew steadily worse at each succeeding grade 
level sampled. 

Many tines the results of educational evaluation nay not 
cause a change in policy i-uT.ediately - as the researcher could 
find desirable. However, nany reconiraendations _ of the TEII-O 
study have been considered and urged for adoption during the 
subsoauont years. For example, higher spending per pupil with 
its accoir.panyina concentration on those most in need has much- 
support. The January 1C74 Presidential message on education 
has strongly urged advanced funding to enable better local 
plannina and staffing of assistance projects. At the same 
time, there have been few recon-.nsndations for cooperation 
from the three groups which should be working cooperatively _ 
to helD the economically and educationally disadvantaged child. 
The health, welfare and educational policy-makers, administra- 
tors and workers should be cooperatively building on each other 
efforts to look at the total problem. 
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PART II: THE STULY 



A. suyyj'iu PR 



This study v;as undertaker, ir: April of ^^(7 as one of the 
major national means of evaluating ccr.pensatory education pro- 
ject srants a*..arded under Title I cf the r^lementary and S-jcondary 
Education Act of IS'65. The tv;o ir.ajcr objectives of this study 
v/ere : 

1. to determine v;het' ^r statistically significant 
enhancement of pu. _1 performance had occurred 
within the Title I schools after the implementa- 
tion of Title I programs, and 

2. to determ.ine v;hat characteristics of the pupils^ 
the schools, and the compensatory programs were 
associated with changes in pupil perf orm.ance . 

Both TEMPO and HrT// felt that the answers to these questions 
would be useful in determining and explaining the success or 
lack of success of these new programs and would influence the 
future design of compensatory education projects. Such informa- 
tion also would be needed for policy guidance for both HEVJ and 
the Congress. 

Title I funds were spread over m.any cities^ schools and 
pupils. There v/ere approximately 9 million students participat- 
ing in these projects during FY I967'. It was imipossible to 
study this entire population. The varied nature and short dura- 
tion of the Title I projects at the time of the evaluation also 
meant that inforn.atlon available at the state and federal levels 
was limited. This led to the need for on-site field evaluation 
and data collection in a sam.ple of Title I schools. Factors 
considered in the selection of the sam.ple school districts were: 

1. geographical and size dispersion 

2. urban versus nonurban 

3. data availability 

cooperation of schocl personnel, and 
5. expectations of successful programs. 25 
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Ihe last 'itei:.. *'oxpectatlor.;: cf su'jcossful prccro...s." '.:a.s 
included sir.ce it v;as araio Ipat^.a tr.at art or only c.r.e ,;ear of 
Title I exposure there raght be ^c.r.e difficulty in fi: j.ing 
measurable evidence of pupil prcrrress. Tr.erefore, KI'. person- 
nel helped selei-t sample school districts consicered :xst lil<ely 
to shov; sigris of progress. Initially fourteen school districts 
throughout the country v;er^ con.-:iaered as sample caruidates and 
visited. Of ti'.ese, three did not have sufficient da'ca for 
analysis, and thus the final sample was reduced to 11 school 
districts . 

VJithin each school district, a subsam.ple of Title 1 schools 
was chosen v;i-ch the assistance of local ad::.inistr8t.crs . This 
local subsarrple included yaryin^: values for each of the follov;-- 
ing characteristics: 

1. Pupil attributes 

a. educational deprivation 

b. economic deprivation 

c . mobility 

2. Cchool attributes 

a . enrollment 

b . rac ial student mix 
a. attendance rates 

3. Compensatory Educatior. program, attributes 

a. miagnitude of prior compensatory education programs 

b. intensity of current compensatory education programs 

The final sample included 132 schools • contain in,-^; 113,000 
pupils from v;::ich pre- and post-tCot scores of 3^K-0^^ pupils were 
obtained and analyzed. 

Joint Tu^^vpO and HSVJ field teamis conducted on-site searches 
v.'ithin school districts for the r.ecessar;. data to coi:duct analyses 
The desired data related to cupi: cerf crraiice , schcc: . ard pupil 
characteristics, and those describ'.r.g cxp^tKi itures for Title i 
programs. Reading perform.ance v:as ch.osen as tr:e prim.ary indicator 
of pupil acl'uevement because it is a skill in which the disadvan- 
taged^ are m/ost noticeably deficient ar.d because it uas :-.r.o;.n th^at 
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ira-7 — "^.ool districts had iir.plo'--r:ted scrr.e form of reading 
inproveir.ent prcgrarr. as a part of tr.e Ir Title I efforts. 

Attendance ani droccut rates v.ere included to test the 
hyoothesis that special ccrr.pensatcry projects rniglit ir.prease 
atte-da'-ce a-.d reduce drooout rates and, hence, contr..ouc& to 
perr"ortr.ance srainj. School and pupil cnaracterist ics -..ere 
thourht to h£ve cotential ir.portance v.-ith respect to F^pil 
aci-ie-er-nt chan-:es and, likev.-ise, it v,-as natural to loo/, f ^r 
relationships beev;een pupil pcrf orr:.=.nce and Title I cxpena i .ures 
per pupil. 

Tv;o >r.ethods v;ere used to evaluate, achievement er.hancerr.ent : 
the "longitudinal." or "individual," rr.ethod and the fi.-.ed-- 
gv^de" rethod. Bo-.h of these focused cn actual changes : ro... a 
pre-t-^-t" and cost-test situation. A third rr.ethod of analys^.s 
utU--2:ed so:r.e' standard statistical rr.ethods to investigate .he 
existence and validity of relationships -oetv.-een observed changes 
in achieve:r.ent and pupil, school, and program characteristics. 

In the longitudinal m.ethcd (see Figure 3) an individual 
pupil's test score before exposure to Title I programs is 
coLared with hi. score after exposure to these CE actiyioie. 
This progress can then be compared to normal fates of ^--^-^-jse 
or ?ne acnievem.er.t gaiu. that -.ould be expected v;ithcut special 
pro-rams. Hcv.-ever, this method requires that com.paraclc tests 
be hven to the sam.e pupil in successive years, and tnis nad 
not been dene in most of the samp.Te districts Furtnermore, the 
test data for eacn year m.ust be ^-'^^ntifiab e_ for inaiyioual 
pupUs. but in reality m.uch of it v;as availaole to xz^.-.tQ only m 
term.s of the distribution of the class as a whole. 

In the fixed-srade approach (Figure 3) a specif ic^grade 
and school is defined as a unit and tne perform.ance oi one pupiio 
in that unit is com.pared in tv,-o successive years. That _ is, tne 
different oerform.ance levels of, say, the fourth graae m a 
certain scnool that had received compensatory education are 
cSmoared vath the perform.ance levels of the f^^^th graae in the 
qa-e <=C'-.oo± for the previous year v;hen no compensatory eau^a- 
lion ;:-as deceived. This v;as h, important m.ethod of evaluation 
for the first tv.-o years of Title I, since the Previous year s 
Class could serve as a control or com.parison grc-ip. ^l^hou^h 
some specialised rem.edial program.s under Title I ccncentrated 
on "nTv a sm.all oart of a rarticular class. thereby_ ruling out 
?f;.^->^4--»aje tvce of analysis. :r.ny other Title i program:5 
such Ts'teacher aides, guidance counselors and l^v.-erea class 
qi-e aff-^cted all m.embers of a particular class or an entiie 
school Sirxe m.ost of the funding v;as spent on general programs 
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Figure 3. CorriOarison of fixeci-^rrade ar.d longitualnal 
, evaluation apprcacir.es. 
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of Titie I ESSA Corr.oensatory Education; Mai-ch 1971 
71 T:-!? 23, uenerai r-lecxricTirrK) , .-.ashington, D.C., 
p. 6. 

of this latter type in the first and second years of Title I, 
it was decided to er.ploy^the fixed-grade method in all school 
districts in the san-.ple.27 

As described in Part I of this chapter additional data 
collection and analysis v;as conducted by lE/.a uncer ^-^^^jl^"- . 
Sn contract frorr. KE^.' (this v;as called Phase ^^^^ , ^nj^^e ^ * 
This additional '..-ork was initiated after tne results o. .nase 
T ^nalrcii reveal-d no significant enhancerr:ent of achievement 
accSSnied the fi?st one^and one-half years of Title I activities, 
assS^iir-the orojects were effective. Two major reasons we e 
adva-co'd of why the Phase I research effort did not actually 
meSure achievLent gains: (1) Title J/^^irt^f,^,^';^ ^^^ieve- 
moderated or leveled out a long-run pattern ^^^eciiri ac.ae^e 
ment levels fr the disadvantaged groups, or ) .^^^^/f i/^;:^";^^^, 
ti-ns b-tv:een exoosure to Title I compensatory ecacat ..on V^J>f-^- 
aid Achievement changes were r.as>ed {or averaged oven b-Y ti^e 
necessity (by data availaoility ) of limiting ue ^''^-^If ^^iJ^^ry 
treating each grade in a sample scnool as a unit ^^^-^ prim ry 
purpose of Phase II was to investigate the ei . e.t o. t.,..<- 
factors . 

During Ph.a.e IT, T^IMPO wont to five f '/^^^'^f ,?^'^j:J?,^ous 
to obtain com.prehensive data on ?-ch ieveme..t ti-nds foi t}.e previous 



26 



five years and on the actual allocation of regular and Title 
resources to separate classrocr.3 (or ctner stuaerit subgroups) 
for v;hich pre- and post-test acr.iever.ent test data had ceen 
studied earlier. Since school districts do not ordinarily 
maintain cost cata at this leyel, 1?v?0 had to do extensive 
allocation of costs based on staff assignrr.ents and pupils affected. 
VJith this additional, rr.ore refined data. TZy.PO v/as able to rnai-.e 
ad.justn^ents for past trends in a district's achievement trends 
and carry out a rr.ore refined ?.nalysis of the associations betv;een 
achieverr.ent changes and various characteristics of schools, 
pupil, an.d Title I treatments. The :r.a;or results and conclusions 
of Phase I remained unchanged after tnio additional investiga- 
tion and analysis. 



SBLSCTIOI: 0? SITES 

As discussed in the preceding section, the data v;ere 
collected from a non-representative sample of Title I schools. 
A. t>;o-sta.cre samipling plan was used: first, school districts and 
then a sample of Title I schools at each district v:ere selected. 
Since this' study v/as exploratory in several regards, it v:as not 
considered critical to have a nationally representative sample. 
It was soon evident that the combination of a six-month study 
and the necocsity for extensive field data collection and test 
data conversion taslc for each site meant that a fairly sm.all 
numiber ox school districts could be included. Earlier hopes had 
ranged up to 100 districts, but the practical aspects of time 
and money soon m.ade a target of 15-20 districts more feasible. 

As a matter of fact, it would have been very difficult to 
select a stratified random sam,ple that would have been nationally 
representative of several relevant Title I characteristics be- 
cause of the lacr: of inform.aticn (at the national^level) about 
specific Title I pro-'ects at the early sta^^es of ilSEA. For 
ex?miDle, it might have been worthv;hile to select a variety of 
types of Tltle^I programs (i.e., reading, teacher training, rem.e- 
dial classes, e'l c » ) . But there was no national comipilation of 
local program activities available by individual districts. The 
sam.e was true of the number of pupils affected by local Title I 
activities, minority pupils affected, etc. The situation wai, 
com.olicated by the fact that ^-^ost districts utilized Title I 
monies for several different activities. Moreover, it was expected 
that m:cst of the Title I activities would end up being carried 
out in variance to indications on the advance application or bud- 
gets. It is im.portant to remember t}":at chis was the first year 
of a comipletely^ nev/ concept of federal assistance and tlvcre were 
still many uncertainties at federal, state, and local levels about 
how the programs would or should stabilice. 
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Kcv;over , ■..•Ithin t:-.-:'se llrr.:.t C:.!: Icr.s , it v;as desired that the 
san.pl-j '^-j x>-0i-e£-i:.iao:7-3 of tr.o r-j-r.^c o:' values for certain 
characteristics of school districts so that the analysis could 
be as s.oulicable as possible. Tneref ore , the, procedures ensured 
that the' 3a:;.ule included a mix of districts (a) frorr. all geo- 
graphical rer-ions. (b) from urban and nunurban areas, (c) from 
a rari:~e of enrollment sir.es, a.-.;i (d) from both city and ccuncy 
types'cf local education a^enci-s. In addition, school districts 
v;ere riOt included unless D:iZVJ believed they had data available , 
particularly student pre- and post-test scores in ccr.parable 
orm) , ana that district persor.nel -.^-ould cooperate v;ith ~il'A?0 
field : nve3ti;:ators . Superimpose a on all these criteria './as 
the L:..:.-; desire to have a fair number of successful (or exemplary) 
comoensatcry education progran.s so the reason for their success 
coui*-! be investigated. "This latter criteria v.'as also applied, 
but '..-itn difficulty since the state and federal levels had very 
limited inform.ation on actual operations or orogress at the 
local level, and because the selection of districts had to be 
completed ■..■ell before the termir.ation of the school year. 

Based on these criteria, EKZVJ and USOE assum.ed the responsi- 
bility of selecting (with the advice of TZ:-:?0) the school dis- 
tricts from ■..■iiich to obtain data for the study. A tentative 
list cf 20 school districts ■.-.•as chosen, .vith the objective that 
"^cro'"" satC'^^' oducat'^cn orogram.s vculd be surveyed in as m.^ny f^f 
those districts as possible -..ithin the tim.e allov;ed for the 
study. 7xtcr.sive visits by Joint rHS/;-T::.y.r 0 teams ■.vere miade to 
l4 districts throusrhout the nation. For reasons of lack of 
student test data or lack of cooperation, only 11 v.-ere used in 
the data analysis phase. Detailed trip reports on all 14 were 
provided to EiiZ'.v. 

Durii c the initial visit to each district, a samiple cf 
schools -..-as^selected from those eligible for ESEA Title I sup- 
port Tn the- i'ollowins m.&rner . Working with the district super- 
intendent (or his staff representative) in each of these districts, 
all T-Mle I scnools v;ere arrayed by the specific pupil, school 
and cor-p-j! -;atory education program attributes noted on page 2, 
Fart ll". 

Fro:^. this cross-tabulated list of schools eligible for^ 
ESSA Titl-' - s\;pport, a sam.ple ■.■■as chosen to include t}:e desired 
wide ran--f' of values for each oi the characteristics. For 
exai-.ole. :r. a given district, several schools were chosen with 
prsdc::.:u'j.'.i.lv _^lac< pupil pcrv.lat\ons but ".•.■ith differetit degrees 
of e:o; cml-- and educational deprivation. 

VMthiin a .^iven set of characteristics, sf " .ti on was made 
without prior kno;;] odce of the 3uc:ess of com.p. satory programs 
at anv of the schools within a school district. Table 1 illustrate 
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the spectrum of characteristics prc-sented in one of the .-air.ple 
districts. A sirr.ilar diversity of characteristics exists in 
other districts. 

Table 2 

CHARACTERISTICS OF SAVPIJ ELSl.ZIITARY 
SCHOOLS Ii: DISTRICT ^. 
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NOTES : 

^Intensiv^e Teacher Aide Program in addition to Title I Teacher 
Aides 

^Special Remediation Program in addition to Title I Remediation 
Programs . 

^Percentile rank: v/ithin District ^, 1964-66. 
^Faniily an.nual in";or:es $2.CCO (1S''65 census data). 
Source: General Zlectrl: T'£y.?0 , Survey and Analy^esi cf Results 
from Title I Bunding for Corr.pensa xory Education, Santa 
Barbara. California^ General^ Electric 67 TMP-llS;^ 19^7 ^p. 5. 

I'Thlle schools v;ere designated for inclusion in the study^, 
for several reasons, the grade rather than the school as a v;hole 
v;as the unit chosen for analysis in the, study. The measures of 
student perf crir.ance used are available for specific grades but 
not for all grades v;ithin a sch.ool. More important ^^co:r.pennatory 
programs are not necessarily spread evenly throughout tha grades 
within a school or school system. Arrregating data on t'r.e resources 
and participation in compensatory prcgra:\3 for a v;hole i'cr:ool 
or aggiegating student performance for a whole school is likely to 
mask the effects of compensatory education. Therefore, data on 
pupil performance., descriptions of compensatory progra^m.s,, and 



exoenditurns for both regular and ccrr.pensatory programs v;ere 
gathered by grade for the 152 schools included in the sar.ple. 

Of the 150 ::chocls originally selected for inclusfcn in 
the sarr.ple^ 132 ccritained one or more grades - distributed as 
shown in Table 2 - for v;hich achievement test data were avail- 
able for the tv;o academic years in question. The sample includes 
the test results of 31^ school grades for each of the tv:o years. 
The total number of students in the sample is just under 3^^000 
for each year . 

Table 2 

DISTR]5UTTG:i OF TEE SCHOOL GR.^.32S USED HI THE 
ANALYSIS OF ACHIS7E..3:;T TSS'I RESULTS.' 
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Source: y.osbaek. F.J., and others, Arvaly^es of _Corr^j)ensa tory Euu- 
'cation in Fi^-'=' Schoo l Districts, voiutr'e jl. aun^rr.ary, 
TtugT"!^-!!^*"; — aSi, sr in.Tfo; . , v.-^ . , Jeneral ii;iectric TiMPO , 
68"tmP-S'3' (1)^ ^'^35. 



C. ANALYTICAL APPROACH AI:D :.:SASlnl-:S 

This study v.-as intended to orovide an estimate of the 
effectiveness of ccsnuensatory education (CE;) programs by measuring 
acaden.ic and behav.-^rial changes of students v;ho have been^exposed 
to these con-.Densatcry prograir.s . An attempt was n^.ace to loentiiy 
the student school, "and prograir. characteristics that v;ere most 
hif;hly correic.tea v;ith such chances as effectiveness. Ihe 
intent was that if a significant improvement was observed in 
a group of students, an attempt v:culd be m.ade to describe the 
arouD and its environment in socioeconomic term.s as well to 
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determine v.r.at Goecific activities produced the change. This 
viev; allov;ed for^ the po3r:ibilit/ cf an interaction bet-.;een 
socioeconcriic groups and responsiveness to various prograrr:s: 
a certairi program or activity may be highly successful in one 
group but not^in another group of a different socioeconomic 
character. 

Initial study plans included the follovn'.ng ir.easures of 
enhancement: achievement test scores, attendance rate, drop- 
out rate, and frequency of disciplinary actions. The effect 
of C2 ;;as to be measured by cc:r.paring results on each measure 
before and after exposure to C2. Hov/ever. because of cim.e 
constraints^ it became necessary "Co restrict analysis to achieve- 
ment test scores and attendance. 

The objectives and m.easures of student performiance used 
in this study are short-range ones; they do not deal v. ith duration 
of changes nor v:as an attempt r.ade to follov: the student's 
future perforn,ance and earning pc-.;er or benefits to society. 
Although these longer-range objectives and associated ir.easures 
would be more appropr late^and revealing as to the productivity 
of C3 program.s / such an approach v.'culd require a much longer 
and di'fferent icind of study. 

One of the key early decisions faced by TEMPO v;as the^ 
selection of a m.easure of achievement for evaluating Ch.. Alter- 
native m.easures for com^parison that could be used, as inaicatcd 
in Figure -4-^ include: 

1. Slope from a to b compared to slope of 1.0 for 
norm, achieving pupil, 

2. Achievement level after exposure to CE (b) co^npared 
to norm.- achievemient level (d)^ 

3. Achievem.ent level after exposure to CE (b) com.pared 

to an estim.ate of the potential level of pupil involved^ 

4. Measurem.ent of the difference betv;een the observed 
value for (b) and the estimiated value of (c) i.e.^ 
estimiated achievemient v;ithout CE."^^ 

In order io avoid measuring the progress of pupils enrolled 
in CE activities in term.s of the so-called norm-achievirig pupil ^ 
it v;as decided to use the fourth m.easure. 

Read in-: achiever.ent v;as -^he only '^acadcmiic" performance 
measure chosen to investigate the first study objective concerning 
the existence of enhanced acn.ievem.ent . Although it m.ay be ques- 
tionabl-^ among som.e social scientists whether this measure should 
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Figure 4. Identification ^o-f different 
measures of achievement. 
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be er.Dhasized In a study of t::Lz type, reading is a fundamental 
skill" in cur socioty. and cnc- in v;hi2n disadvantaged cr.ilaren 
are r.ost noticeably deficient. It is also a sxill which affects 
perforrr.ance in ether academic areas. Although it v;ould have 
been desirable to consider oi;ncr acader..ic areas, the tirr.e avail- 
able for this study did not perrr.it this extension. 

A variety of characteristics v:ere :r.easured and analysed to 
investigate the second study cc:ective. The behavioral character- 
istics planned for analysis ..ere (1) attendance data (2) rate of 
dropoing out of school, and (3) change in i^.-^jix::oeT of disciplinary 
actions. Attendance rate and change in attendance rate -..-ere the 
only n-.easures for -..-hlch suffic:-cnt aata -..-ere available. It v;as 
found that, in general, these types of behavioral data '.;ere not 
readily available at tne school district level and had to be 
obtained at the schools therr.selves . Program cost is an i:-.portant 
program characteristic v.'crthy of special mention. It v;as (and 
has since cecom;e even miore of) a source of controversy in the 
achievement of students. The basic viev.-point in defining cost 
data requirem.ents is the need to relate the real (physical) and 
money resources of a compensatory education program to measures 
of effectiveness (student reading achiever.-.ent ) . Since both the 
absolute level and the incremental change in effectiveness 
resulting from compensatory education ..ere examined, the absolute 
a'^d Increm.ental com.oen??.t!~''"y v-o ^ on =.nd funds v;ere also re- 
quired'r Therefore ," the cost.- of regu.lar school programs^ before 
and after ccmxsnsatory program.s as '.■;ell as the costs of the 
comoensatory prO::rams v.-ere required. Cost categories were defined 
within the context of financial accounting proceaures reccmjnended 
to school districts by tho U.S. Office of Education in its Hand- 
boo'^ II. Since costs at the Individual school (or indivicual 
grade) v;ere usually not availacle, TLVPO had to do considerable 
cost allocation and estim.ation. Costs for Title i com.pen.-atory 
education .activities had to be related (or allocated) to the 
specific period betv.-een tne pre- and post-test m.easurem.ent s (which 
was often* different for each grade studied). 

Original plans to obtain data on pupil perform.ance and 
exposure to CE '.vere exoanded to include information on the charac- 
teristics of the Dupiis, their schools, and their comjr.unit ies . 
The larse variation in these later variables was judged as imiport- 
ant for~exDlaininr: observed changes in pupil perform.ance between 
1965-66 and 1966-57. 

The analysis of ach icv£;\.-'nt test results employed a "fixed- 
grade" aDoroach in m.easurin.^: t).e effects of compensatory programs. 
The distrioution of achie /e:..=nt scores for children in a particu- 
lar --rade a>^d school for the year preceding exposure to com.pensa- 
tory'education -.vas com.pareii .ith th:e dlr-tribut ion of ach.iever.ent 
scores for a, different group of children in the sam.e grade and 
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school ir. the f cllov.'ins year, v;ner. ccir:pe'-:catory education pro- 
grs.rr.c i.ad teen i::.ol£;r.jr.ted . 't v.-as as.-5i;:.'.er.; that, Ir. t: o 
absence of soeciai coir.pensatory prcgrarr.s. the pattern cf 
achieverr.ont scores in a grade v;ould rerr.ain constant frcrr one 
veer to the r-p-t. Thus, chanrec in the dir:tributicn of test 
scores betv.-een ^he tv:o years .;cu1g reflect the irr.pac'c ci partj- 
ci nation by the studet:ts in cor.pensatcry programs, me useiul- 
nfs-^ and validity of the fixed-grade approach v;erc- governed by 
the^ability to identify and cope v;ith the additional factors 
introduced by comparing results for tv;o different sets of 
pup-" Is, That is, anv changes in the achievement pa-ttern from 
one year to the next" must be evaluated in light of normal 
changes in the level of educational services providec oy the 
school district and changes in the socioeconomic composition of 
the student oooulation. Either or both of these factors can 
influence the academic perform.ance from ore year to the next of 
the pupils in a grade. 

The miost important uncertainty in relying on observed 
differences in test scores in successive years to assess the 
effects o^^ compensatory progra:r.s arises out of the possibility/ 
of trends in achievem:en^ scores v;hich are independent of com.pen- 
satory education. There -.vas reason to expect a dov;nv.ard trenc 
in the achievement level of pupils at jnner-city schools^ (v.'hicn 
o^^e+^-+i'ro T-ocjf of fhp Q^niDTp^i relative to the entire nation. 
This "decrease 'in achievement scores for a given grade level over 
time has been associated with changes in the soclo-^oncmic com.po- 
s it ion of the student population. 

An alternative approe.ch involves the analysis of changes 
in the achievement test scores of the same students or groups 
of stude-ts through a given time. Longitudinal analyses are 
attractive because som.e variables are h£]d constant. In this 
stud^s the choice of the f i:-:ed- rrade approach was dictated oy 
th^ difficulty in obtaining substantial am.ounts cf longituamal 
data Schoc systems do not as a rule tost each grade each year. 
Further, hi.?h oupil m.obility, the absence of centrally m:a:ntained 
cumulative records on individual pupils, and the difficulty at 
the tim.e of the study of linking a pupil's test scores to his_ 
participation in particular coi.-ipensatory programs rulea out one 
longitudinal approach. 

Most of the analyses in tl:is study were based on test^data 
for academic vears 1^6^-66 and 1966-67. The form.er (IrcS-cc) 
was considered the "ore" year, or the year prior to -c-.nensatory 
education orccrams , and the latter (l^^cc-cf), trie posi year, 
or the year in which there had heen some exposure to compensatory 
pro-ram.s. Although study tears collected inform.aticn on compensa- 
tory programs regardless of tl e source of funding, tne major 



ERIC 



34 

-33- 



infucicri of funds resultoo frcrr. Vltle I. Funds frcr. Title I 
did r.ot reacr. z^rxol "y?i-:-:'3 u-ti.l f r e coring of V-.t. and 
in r.any cases the activities socnnr.red und3r Title I '3 id not 
be?ln until the end of the school year. It v.-as assured , there- 
fore, that tne benefit s of corr.penc-atcry progra-s v.'cu.l d net begin 
to be evident in achieverr.ent test score:- until the l^ct-tT 
school year. 

Relatively fev; school syster.s keep records on the amount 
and ty'je of co:r.oensatory nrorra.T.s in specific grades at a school, 
nor v;as it usually possible to identify the specific students 
within a grade v.-ho v.'ere participants in corr.pensatcr- progran-.s . 
Consequently, it v.'as decided to use test results for all pupils 
in a crade and to rather information on all grades in the 
sar.ple schools v;hich had •^e-'^-n tested v;ith the same achieverr.ent 
in both ls65-Cc and 1966-07. 

Various standard analytical and statistical techniques 
were used in investigating the data collected. Vfnen the exist- 
ence of achie>/e:r.ent enJ^.ance:r.ent v.-as -anined using the 51^ 
grade observations (or subsets there- ). it v;as necessary to 
also checlv the statistical significance of the resulting statis- 
tics since a sarrpling process v.-as clearly involved. Comparison 
with the standard error and the rele-'ent underlying statistical 
distribution allo-.;ed deterr.ina-t ior. of the probability that the 
obser'-ed sample result could have happeni:d by chance, .analysis 
of variance (and the r.ore discriminatl.-.g analysis of co-.-ariance ) 
was used to test if there '..-ere sig:) if ican.t differences in ob- 
served changes in achieve.-.ent test scores among school districts. 
All ^^->ese analyses v.-ere coni-acted for t;.ree n.easures cf change 
in 8chTe-.'cm3nt--t:'ie mean, the first deci?.e (10 percent point) 
and the first quartile (23 percent poir.t) of the distr ibution 
of the performance of -the grade. 

In 'investigating: the association of various school, pupil, 
and env^ronrer-T.pi. characteristics -..'ith enhanced achievement, 
se-veral additional statistical methods -..-ere used. Such a situa- 
tion is analvtically difficult because of the lil^elihood of joint 
and interactin.- actions of many variables. The f ollo-,;ing _three 
types of analyses v;ere carried out in a; atter.pt to identify 
the relationships betv:een the variables and the changes in 
achievemen.t 

Simple correlations bet-,':een changes in achieve- 
ment an.d each -'-ia;:'! e -..ere cc: puted from the 
cor.bin.ed data or. all distrijt.: and separat.oly 
from data on each cf the 11 districts. ahis 
procedure identifies the degree of association 
betv;een any f.vo variables. 
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Ar.alysis of variance .;as computed for the cnans^ 
at tr.e mean, first douile ar.d firct quartile 
using all 31*^ observations. The total group 
v;as partitioned according to contrasting levels 
of the variables to ectirr.ate the effects of these 
factors . Analys is of variance is helpful in 
detecting significant differences among speci- 
fied sets of results, including groups thax 
cannot be described in numerical terms (such as 
am.ong districts) and therefore is useful v;here 
a correlation coefficient is not applicable. 

Multiple regression analyses were done using 
a number of variables for v/hich data v/ere 
available. Multiple regression is helpf\u for 
identifying the relative effect of each vari- 
able on changes in achievement and is helpful in 
reducing tne sam.pling variation v;hen testing for 
significant effect from com.pensatory education. 
It is especially helpful v.hen th.e variable of 
interest has been affected by many different 
variables. In constructing a structural model 
to use for this li^'-ea^r regression analysis, 
TEMPO postulated that the following five vari- 
ables were significont in explaining observed change 
in achievem:ent J and then estimated all the 
multiple regression coefficients. 

(a) average Title I CV program dollars per 
student expended by the post-test data 
for all students in Title I schools in 
the district 

(b) achievemient level at the level at the 
beginning of Title I programiS 

(c) mobility in and out of the school 

(d) percent Black students in a school 

(e) attendance rate in a school. 

The dependent variakle, i.e., the one whose variance 
was being explained in term.s of the above five 
independent variables, v:a3 alv.-ays the cbser/od 
change in achievement (separate regressions v:ere 
comi^ieted for both the ciia!*^.ges at the moan and 
the fii-st decile. 
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D. FIELD i:i7;3TiGAi ic::s 

All data -.-.■ere collected tnrcusT. visits by pro.ject person- 
nel to the school districts involved. Visits ra:i|ir.? frcm^ three, 
days to two v;ee:<s In lengoh were .T.ade by Joint DHiiV.-TiivPO teams 
to each ci" the 1^ districts. Th-'; ori^.ir.al contact witn tne 
district and reauest for data was r.ade by DHEVJ. The original 
irroetus for such on-site investigations was the shortness of 
tirne--(l) there was no tirr.e In a si/.-rr.onth study to go tnrougn 
a T-^n-thy survey design orocess a:.d a.. ait rr.ail replies, and 
(2rthe contract starting date of April IS'C? left little time to 
get data frcr. school districts oefore they closed in vay or 
Jur". The only practical aoprcacn •..•as to establish data reo.aire- 
ir.ents (along with operating' guidelines ) and get professional 
staff r:embers cut into the field very quickly. 

Similar field work: proceoures were utilized in Phase 
where additional data for in-depth analyses were obtained from 
five of the original districts. _In the spring of 19c5, joint 
two-person teams from Diir.: and Tr-Xi-O spent approximately 
one week at each of the five districts. 

:\o forr.al survey instruments were developed for use of 
resDondents, since all data collection was by personal visits 
a-nd' interviews. Mo-covcr, the number of cchccl districts and 
schools was small enough that this aspect of obtaining and nandl- 
in^ data was manageaole witncut significant problems. Tiumerous 
data collection fcrm.s or guides were developed for internal use 
by field team rem.bers to facilitate collection of recuirea infor- 
mation. These identified required data eler.ents so all districts 
would be described in consistent .■.■ays. Although the design of 
these many forms .-reatly helped define and structure cata require- 
ments, they did not turn out to be as useful as anticipated. 

The difference in availability of certain types of data, 
level of detail, record-keeping systems, and the use of autom.ated 
records n-ade it difficult to complete these forr.s for each sample 
schoo3. They continued as the basic guide for information being 
sought, but studv team members found it more efficient to devise 
variations which facilitated obtaining data in the several school 
districts . 

In general, it was not necessary to transcribe reading 
achie-emert data on to a star.dard form. These data were obtained 
^r— ^ rarv o'- the districts in the form cf copies of reports or 
duDl^-cate ccoies of m.achine tapes, machine printouts, ana nano- 
written records. Cost ana resource data for regular scnool pro- 
grams i'-' some instances -..-ere autom.ated and in others were 
transcribed by hand. The annual budgets, annual reports, or 
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other operating plans were brognt fcacl< by ea-jh fieJd t^am and 
often t rov'idei an adequate scurce for jo.ct/resource c^ita. It was 
soofi clear ti.e ori^ir.al applications for Title ^^unain.- -..'ere of 
littli. use since actual programs differed so much frc;r. submitted 
project plans, (due to 3ate starts, revised plans, reduced 
funding, unavailable staff, etc.}.' 



asse;.:3Li::g ai.d frccessi::g the data 

The necessary data cn Title I CE programs, student per- 
formance, and district characteristics were assembled during the 
three-day to tv;o-wee:< visits to each district. Undoubtedly 
m.oro time would have been i:elpful in every case, but the need 
to visit many locations in order to expand the samole sizes of 
schools and programs was considered more important". Cne nega- 
tive effect of ti.e intense travel schedule was the irar, ility to 
surrnarl-e and analyze adeo.uately the inform.ation and lir.ore ssions 
after each trip. 

Problems encountered in collecting and -processing the 
information provide insight into the complexity of such a study 
and are, therefore, useful bacKgrcund inform.ation. Briefly 
stated, the more severe proble.ms were 

the amiount of derailed in.'ormaxion required for 
making performance m.easures com.patible v.'as very 
great and several special conversion factors were 
required ; 

achievement scores cam.e from different tests ana 
from tests whicli were administered at different 
points during the school year am.ong and within the 
sam:ple districts; 

inuoh of the readily available data on ccmpen.: atory 
education were not in usable form for study because 
they did not give inf or:r.at ion for specific grades; 

it was difficult to distinguish betv;een corrcon.satory 
education and regular school programs and between 
compensatory education prograir.s funded by feceral 
and r.onfeaeral agencies; 

the large transfer of pupils into and out of a 
sarr.ple ^-^ch.ccl -.a:ic it difficult to identify t:.o 
amount of ccr^:pencatcry education to which pupils 
had been exposed. 
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In its ::achir.,i^tcr. Offic^:S, TZyi-r processed ar.d ar.alyzed 
the oa-ca it assv-^-lcd fror. \zr field : :/.-estigat ions . :*ar 
the ^rreatest efi'crts ar.d prcclarris a'jccr.panied the use of a::hieve- 
mer.t^tests to rr.ear-jre the cha.-.re of performance of suuder.ts 
exposed to Title I proe'ran.s. 'fnere -.as no opportunity, lice, 
or ir.oney to adrninister^^s irnilar cr ccrxarable t sts for purposes 
of this evaluation (for exar.ple, tesx:^. of status before treat-- 
ment v;ere nco possible, si?:.ce zr:e prc^ran-.s v;ere almost finished 
before the study smarted). Therefore, it v;as necessary to 
ma'rce best cossible use of standardized achieverr.ent test results 
already adr:;inisrered by school districts. Utilizing existing 
test data created other significant proolems and requirements 
v;hich are discussed in the foilov;ing four sections. 

1 . Processing Achievement Data 

There are tv;o main aspects in obtaining com.parabls units 
for observed differences in test scores. One is comparability 
among different tests and different test dates v/ithin an academic 
year. 

The com.parability over different achievement levels v;ithin 
a grade was obtained by converting zc the Standard T-score . -^-i- 
The scale for tr e Standard T-score has been constructed so that 
a change from. 3) to 35, for exam.ole. is comioarable to a change 
from 65 to c^-. Com.parable in this case means that the amount 
of effort in C3 require a zo raise the achievemient level five 
points is approxim.ately the same in both cases. As a result^ 
T-scores can be averaoed and subject ea zo statistical analysis ^ 
whereas percentile scores, for exam.ple . cannot. In some of the 
analyses, results for differ;^nt grades were con.ibined, and in 
these cases it was assumed that differences in T-scores are also 
amiong grades. That is, the amount of effort in compensatory 
education required to increase an achievement level from, say, 
30 to 35 in one grade is approxim.ately the sam.e as for a similar 
movemient in othe^ grades. 

The second aspect of comparability among different tests 
and test dates enters through the use of the appropriate publishers* 
norm tables for coir.puting percentile scores' for the testing situa- 
tions involved. The T-score is obtained by converting tiie raw 
score for a specific test zo a national percentile ranlc and then 
converting The percentile rank to the standard T-score. The 
assum.ptions used in computing percentile scores were: 

1. Rcadin.g, paragraph ::eaning, arid com.posixe sublcots 
m.easure ' ti^.e samie pupil attribute, 

2. Publishers" norm pcrulationr for each test are similar 
with respect to the" distribution of achievement levels, 
and , 
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3. Publishers' norms pro/ide proper adjustment fo^r 
the tnr.e of year at -./r.ich the test is give::. 

The first assumption w-as necessary because each of the three 
subtests v;as used In one or more sci^col districts. Since the 
prirr.ary Ifiterest '.:as in analyzing differences between pre- and 
post-tect scores it v;as not necessary to assume that norm 
populations were equal in absolute levels but only in the range 
and distribution over the range. 

One feature of the data deserves special notice. Since 
pupils in eligible Title I schools v;ere theoretically selected 
for being economically and educat icr.ally deprived, their achieve- 
ment test scores tend to be lov;er than tnose of the norm popu-^ 
lations. Figure 5 illustrates the distribution of achievement 
scores of a typical fourth grade in a sample school in 196^;-6c, 
E^gh^y percent of the pupils in this grade received test scores 
below the m:ean score of the pupils in the norm population. 

The number of pupils in each of the 31^^ basic ccser vat ions 
in pach of the years 1965-66 and 1966-67 varied from I6 to 
593. This raises the question v.hether each of the 31-^ observa- 
tions should be given the same weight in judging the magnitude 
and statistical significance of average change in test scores 
betv;een the two years. If longitudiral data (i.e. . test scores 
for the sam.e pupils in both years) had been used, the obvious 
choice would be to weight eacn pupil equally. However, in the 
*'f ixed-rrade" aporoach there is no unequivocal v;ay to match 
individual pupils in I965-66 with those in I966-67. The choice 
v;as m.ade to present two types of statistics ^ namely^ 

1. unv;eighted averages and standard deviations of the 
314 ba3lc observations^ and 

2. * v/eighted averages of the 31^ ob:':3er\'ations based on 

number of pupils in each observation. 

In general terms, both these situations produced the same results. 

The technical approach called for analysing each of the 
31k observations separately if statistically reliable result^ 
could be developed. This would avoid the problem.s irJrierent in 
averaging data from different tests and across different grade 
levels. It soon became apparent that ti.is approach was not 
appropria^te and data would have to be pooled. First, t::ere is 
considerable variation in test scores between years that cannot 
be Gstim.ated from available data on the variation among pupils 
v/ithin a year. For example.^ the test could be given under very 
good conditions one year and poor conditions the other year. 
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Cecond, the carr.pl ir::; variaticr. Ir. test .-^ccres v;as so Ic.rge that 
fe . of the OL::or /ei ^ :i:'::oi'-..::os pro- i.:.d post-tcjt 3 :croo 
for individual ocsc-rvatioriG v;ore statist I'jaily sig::li': oar.t . 
Therefore, the s'jorer: vere al]. 'jcr.verted to Standard T-socres 
so that observations could be pooled. 

2, Type of Test 

Data ar.a.lvsis v:as lin.ited to stardardi::ed achievement 
tests adrr:inistered to entire -rades of pupils in a school dis- 
trict for each of tv;o years ^ i9-c: -cc and 19cc-o7. in the 11 

Fi.-^ure 5. Distribution of a jhieveir.ent schores in a typical 

Title I school co-.pared to that for the entire nation. 



69 PUP'LS IN A 4tn G5,ADH IN A 
TYPICAL SAMPLE SCHOOL 



- NATION 




STAND '<0 T SCO^.E 



Source: General Electric TE^PO . Survey and Analyses of Results 
fr om Title I Funding: for Cori pensatcry j duca 'c lcn y 
.Santa baroara. ualifornia^ General ileccric c7~TMP-115^ 
1967, p. 19. 

school districts, five different tests v;ere used: Stanford 
Achievement Test' (SAT), y.et repel: oan Achiever.ent Tesrs {yAT), 
Iowa Tests of Basic Skills (11:::;), lo,';a 'iests of Educational 
Development (ITED), and Sequential Tests of Educational Progress 
(STEP)". The variety of tests and subtests in different grades 
in the 11 school districts is s]xC\<\\ in Table 3* 
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There v;a3 'jo-.s IderaolL, yariatlcn in the time of the year 
in v/hich tonts v.'o^-e .^ivep. or:. or." and e'/en v;ithin school districts. 
Thic resulted in d : f I'erer.ces in arr.ounto of ccir.pensatcry educa- 
tion to ;;hijh pupilc v.ere e::por.ed prior to the lSc6-67 test. 
Such dlfforc.r-es In exposur. tc ccrr.pencatory education cannot 
easily he tahen into account in a simple cor.parison of pre- and 
post-test results^ but can bo incorporated in the regression 
analysis . 

A nev; variable v;as constructed to sliov; v;hat is called 
"effective Title I dollars'' per pupil. This v;as computed by 
adding the Title 7 funds spent in is6--c6 and the fraction of 
Title I fur:ds for lS'^c-67 spent up tc the tirr.e of the 19cc-67 
achieverr-errc tes'cs and dividin^T tine su:r. by total number of pupils 
in Title I schools in each district. This nev; variable v;as 
included as one of the deterir.ining variables in regression and 
cinalysis of covariance. 

Differences in test dc-/ces betv;een 1965-66 and I966-67 
required use of di referent ncrr;. tables for pi-e-and post-test 
or ad just:rients v;i'chin a particular norm table. 

3. Specific Grad es/ Students Z.xpcse d to Title I Programs 

Relatively fev; school systems keep records on the a:'-o\)nt 
and type of CE programs at specific sci.ools. This m.eans that 
it v/as often not possible ;:o identify precisely v;hich rrades 
in a school received Title I pro.eram-5. Furthermore^ it v:as 
not possible to identify the specific students in a grade v;ho 
received C;] from litle I programs. Consequently ^ it v;as decided 
to use test scores for an ertire grade and to use all grades in 
Title I designated schools for v:hich test data were available 
for both lS'C'3-66 ard IJcc-cV. 

4 . Data Calculations 

To aid in the extensive and repetitive processing of the 
large volume of student test data^ T?J/.rO made use of cor.puter 
equipment. A t im.e-shar ing term.lnal v;as installed in the lEMPO 
offices .'.'hich allo::ed continuous connection to a General Electric 
235 comiputer- Frc grams (written, in BASIC) v;ere written to process 
the test s;^ores and calculate various summary statistics for each 
grade unit. The m.any regression and correlation analyses v;ere 
run in a batch mode on. Control Data equipm.ent using the "5io- 
medica.1'' package o:' statistical programs. 

Data received fro:.: sci'^co] districts v:ere in one of the- 
following forms: :^]ass listir;;^s of individual pupils, pun.ched 
cards of 'ndi^'iduai pupils^ cor.puter tab runs by individuals. 
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and oorr.puter printouts of frequency distributions. Test 
scoro^ inolvd":;d cno or rr.ore of tho fcllcv;ing: rav/ scores, 
standard scores (for specific type of test)^ grade (or grade 
equivalent) scores, percentile scores^ and stanines. 

All scores not in national percentile v/ere converted to 
national percentiles by usin? the appropriate conversion table 
provided oy eacn test publisher for each separate fori;, of test. 
Percentile scores v;ere then converted to Standard T-scores. 
The distribution of Standard T-sccres in the population is 
assuir.ed to be ncr:r.al \.:izh a mean of ^0 and a standard deviation 
of 10. This assun-.ption v;as utilised in converting national 
percentiles to Standard T-scores. Detailed conversion tables 
and exaniDles are contained in tne referenced Tr.MPO reports and 
the interested reader is referred there for further detail. 
Figure 3 preser.ts an exarr.ple of the corr.puter output prepared 
for each of the 33- grades. Test results for each grade in 
each school ror 19c 5-66 and 19-6-6? v;ere suirmarized in terms 
of 19 statistics (see lov;er part cf Figure 6}. The central 
part of Figure 3 illustrates the conversion from rav; scores 
to Standard T-scores. The su:rjna.ry statistics are all in terms 
of T-scores. The examples show 5^ pupils in a grade; in the 
overall samole , the number of pupils in a grade ranged from l6 
to 595. 
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Figure 6, ExarrxOe cf data prcce>:oir'g and surrr.ary 
of statlr:t:cc cr eacr. rrade for l^-il^co 
and IS'oo-'w/'. 



Grade 



Post Year School 



Data ip/Rav; Scores to Ru^i Thru Time-Sharing Coir.puter 
Stat Lstical Program 



33 ,3- .^^2 ,3o ,c£ .CS 



Rav; Scores :<anl<ed in 



"5 " 



^9. -2, 30, 35, 33. 



^zcevioir:^ Order 



30 


30 


30 


30 


30 


?? 


30 


32 


32 


33 


33 


33 


33 


36 


36 


^ 






39 


39 


39 


^1 


41 


41 


ill 


hi 


41 




Lo 


42 


42 


44 


Lh 


42i 


LiL 




46 


46 






46 


is 6 


46 


47 


47 


47 


47 


.IJG 


49 


4c 


Lc 


51 


51 


51 


52 


52 








S6 




57 


57 


59 


59 


60 


60 


62 


62 


62 


62 






66 


67 


71 


76 


80 


63 


&5 




65 




105 










' Sc 


ores 


Con 


/ert 


ed z 


c S 


tar.dard T- 


-SCO 


res 


and 



26.7 
29. 5 

34^5 
36.1 
37.1 
38.3 
48.8 
43.6 
45.^ 
48.1 
58.3 



Surx-iary S' 



2o . 


7 


2c . 


7 


26. 


■7" 

/ 


2 c 


.7 


26 


7 


2o 


.7 


26 


.7 


31. 


2 


31 . 


2 


31. 






r 




5 






31 


.5 




5 








J 






3c 


.1 


36 


.1 


36 


. 1 




1 


36. 




36 










. 6 


36 


. 6 


36 


.7 


37 


( 


37. 






3 




• 3 




.3 






33 


.3 


33 






0 


39 




"^^ c 




3 9 




3^' 


. 5 


40 


.3 


r\ 

-- 




•T vyf 




41 




ill 






. c 


42 


• ^ 


42 


.3 




.6 


43 




1, 1 




. lL 






"7 
■ ( 


b 

* r-t 


.7 


45 


.4 




.4 


46 


. 




• ^ 




r 
• 3 




. 5 


47 


• 5 


48 


.1 




1 


50 


• I) 


L2 








c. 




57 


.7 


58 


.8 




.8 


59 


r- 

■ 9 























J. ^ ~ u . 

. o 1 S o , 



Ln Standard T-Scores 



ERLC 



Smallest Varlate 
Lov;er Tec lie 
First Cuartile 
I'lodia!: 

Third Quart lie 
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= 21, 
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p. RESULTS 



This section presents scrr.e of the r.ajor results of the 
study. Since the purpose of this publication is to present 
evaluation approaches and experience rather than to discuss 
particular substantive content ^ only highlights of data results 
are presented. The reader interested expecially in Title I 
ccT.pensatory education per se is referred to the referenced 
publications for a full discussion. 

The TEMPO study involved 35j>000 students and 31^- grade 
observations. On the basis of this study certain characteris- 
tics of the Title I prograrr.s could be ascertained and are irri- 
portant to consider as background for the follov;ing discussion 
of an^jlytical results. Such descriptive information v;as an 
important result of this study since previously there -..-as a ' 
laclc of first hand coir.parative data on hov: the districts v;ere 
actually utilizing Title I funds and v:hat they v;ere accomplish- 
ing v;ith this resource. 

TEMPO found a wide variety of programs being implemented 
v/ith Title I funds. Many of the" districts had utilized a large 
amount of their funds to reduce class size or to provide teaching 
aides; m.an*^ of the Title I programs v;ere basically benefiting 
r% T T n v>^ A fpn + n.^ T so^oo2c '.'/""t^C'it "^^egard to v/hether the 
recipients v;ere specifically educationally disadvantaged. 
In many caches , Title I money v;as spent on equipment or school 
construction, especially during the first year of operation. 
The expenditures per pupil ranged from less than $25.00 to 
slightly over $225.00; the average expenditure v;as $^0.00 per 
year. On the basis of 200 school days per year, this is an 
average of 20 cents per day per child for compensatory education. 
This type of average expenditure ;;as so m.inim.al that it raises 
definite questions^about the, acnievement gains which should be 
expected. 

1. Achievem.ent After Compensatory Education: 



The results to be discussed first concern the initial 
objective of the study: 'whether enhancem.ent of achievem.ent 
accom.panied the extra or com.pensatory education activities pro- 
vided by Title I. 

Enhancer.ent is defined here as the difference betv;een 
achievement level after exposure zo compensatory education pro- 
gramis and the achievement level v:hlch would have been expected 
in the absence of such programs. Vihiile it is possible to m.easure 
the achievem.ent level of the sam.o pupils in the absence of com- 
pensatory program.? had to be estim.ated at the time of this study. 
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The result- of rr.orc of tr.e statistical tests were based on 

ince •oet"eer;acVie;e..ent -•ith and •,:itr.cut exposure to ecrr.pensa- 
torv crorrarns. The operative assurpticn ••■.•as uhat tne diotri 
bSuoK of achieve::.ent scores should not change from one 
to the next (i.e. no adjustr.ent v.-as rr.aae m r^nase I x or possible 
trends in the level of achieven.ent v;ithin a geaae). 

Althcuvh there v;a3 no indication of general f^l^'^:"^..^. 
in the average achievsr.en. level of the overall ^f^.^f'L^.'^veS^? 
in t'-- '-^1^ scv^col :=-rades analvzed, there v:as a slignt irr.prcvem.nt 
In Se achilve^.ent-^evel of students, at the _ lowest achievement 
levels (1st decile) in their respective graces as snown.n Table 
t Th4r-tically ihe Corr.oensatory Education programs snculQ 
h^ve Uen focusing on the most seriously d-advantag^ stuoents 
or those with tnc lowest achievement scores: thereiore tn.re 
was an expectaC Ion of m.cre favorable results for students at 
the lower achievement le\els. 

There was a slight decline in achievement at the mean 
and the first and third cuartiles for all of the grade levels 
in the sam.ole schools. These results suggest that even with 
compensatory education programs present, tnere is a slis-t _ 
negative trend in acnievement level in wnere x^o^^^ 

money was beir- concentrated. Table 5 incluaes resui.s .rem 
SS?h weighted Knd unweighted observations: both approaches lead 
to the sam.e ccncli:sions . 

Several interpretations of study results could be niade 
copc-rn?ro- t^e oostiVe change in the lowest decile, ^ven though 
ihe oosut/e result was not^^ery large (it represented only about 
one-fourth of a month in terms of £"^^^\eouxvaient units ), 
it becam.e'r:ore signif ican.t when contrastea •.. itn tne neg ative 
changes in the m.e.r and at the first and tl^^ird auartiles^ I^^ 
the absence of special compensatory prograr.s, it.v.oulQ ^^J^^^^y 
be e-.-occtcd trat the four statistics woulc m.ove the same 

they were derived from tne same distribution 
ofac?.Ievem3nrscorL. The probability of the observed ^ovement 
occurr^n- at the lov;est decile and the ooserved decline at the 
mean InS-^first and third quart iles would be unlikely by chance 
alone. Therefore, this cou.ld be an inaicatxon of a f^-^.^'l 
effect of com.oensatory programs if one ass ,.r.es they aie ..sualiy 
concentrated on tnc lowest achieving students m a graae . 

Conversely, it must also bo recogni-ed that in seme of 
the sample grade units the test score defining the lowest decile 
^^^.r2 fi^st y-- was ecuol to the minim.um oossible score 
ihus7in s^me' ca'ses, i : 'was im.rossible f or pupils at the lowest 
decile to register a lower score the next year, even i- a 
negative change aid in fact occur, 'inerefore, m tne ii.-.ea- 
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jrrade ssrr.ole studied, the teir.ptaticn to ccn.jecture that Title I 
caused th-i ir.r.rcver.ent at the lov/est decile ir.ust be ter.pered 
by the need fcr a ccrreopcnc ir.g causal explanation cf the nega- 
tive changes in the other statistics. It also must t-i r.oted 
that even^thou.n nine or the ele-.x-n districts shov;ed a positive 
change at the lowest decile, as shown in Table 6, only one of 
these changes -,:as statistically significant at the five per cent 
ccnf idence^levei . There appeared to be significant differences 
amcnc; the districts; hov;ever^ sorr.e cf the differences seen in 
Table 6, (such as in thr ^-.20 change in i^istrict c) can be 
explained by oo. s:.ble .large sampling variation due to the fact 
that there v;ere only fov^r observations. 

The san:pling variation for the mean is smaller, and 
again in Table' 6, it is found that three of the 11 districts 
shewed a sirnificant change at the five per cent confidence 
level. This Table also shows that there are significant differ- 
ences amon^ the districts when considering the mean. The range 
between the l.lo chans:e in the mean for District 13 and the 
-1.35 change in the r.ean fcr District 2 is large. This repre- 
sents a difference of approxima-cely 2.5 grade equivalent months 
over a 12 month period. 

Longitudinal analyses of individual pupil perform.ance 
aiiierences were conduc-ced on a m:ucn amallei- scalt. Tr:6se 
results were able to be obtained fcr only one district but 
were encouraging and indicated that seme Title I programs m.ay 
have orod-jced significant achievement gains which were obscured 
by the preponderance cf the very general ccm.pensatory education 
prelects or by the fixed-grade m.ethod of analysis. However, 
utilizing a longitudinal method of analysis rather than fixed 
grade with a Title j prc^"ect which provided a librarian or 
audio visual eoipment for a whole school or grade, would not 
have been m.eaningfui . Within the district where longitudinal 
analvsis •..■as cossible, it appeared that the gap between the 
educationally' disadvantaged pupil and the norm achieving pupil 
was being su'ostantially narrowed. The compensatory reading 
program involving 790 students vras studied utilizing the longi- 
tudinal technioue. 'it was expected that the students in this 
program would achieve .? of a year per year of school; however, 
the actual achievement increase for students in this program 
was l.ii years in a school year. It is important to note that 
another Title I program w:.ich .vab studied utilizing the longi- 
tudinal m.ethcd of anlaysis and •..•here the expenditures were 
in v-xcess cf the compensatory rcadin-g pre --ram above djd not she,' 
positive or •. nexpected increases in achievement. 

As previously described, subsequent additional investi- 
gation (c.^.lled Pha3e 11^ -..-as autncrir.ed oy the Office cf 2du- 
caticn in order to determine whether further refinement cf the 
Title X data in Phase I of this study would pre duce results in 



Tabic 6. Average change in mean reading test scores 



by district. 



l.uol District 



No. of Grades" 



Observed 



Averogc Clwnge 
in Mean° 



Average Cncngi 
in Lowest Decile' 



2 
3 
4 
5 
6 
8 
10 
12 
13 
14 



39 
56 
42 
27 
28 

4 
55 
13 

4 

32 

18 



+0.25 

-1.35^ 

-0.34, 

-0.16 

+ 1.28 

-0.37 

-0.21 

+0.42 

+1.16^ 

+0.01 



+0.05 

+ 0.04 

+0.22 

-0.68 

-0.70 

+4.20 

+0.26 

+0.55 

+1.78 

+1.32' 

+0.W 



Notes: 




Satislically significant at the 5 percent confidance 



Mosbaeit, E. J., and others. Analyses of Ccmpensatcry 
Education in Five School Districts, Voluir.e I, Sunimary 
Au'-.'ust 1^68, VJashinstcn, D.C., General Electric TEMPO 
68"tmP-93 (1). p. 30. 
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involved ^iv^^^t of P'^^'o'-!, ;ru*5?J?;,„^-'-"= '•^-^her 

w,u:: true m the schools studied bv T-v-o t?e i':;-f -i-'-^''^*' * 
picture on trena, partly because sarolir- variatnon --r;^ r/jL 

fiyrsssoi dVr^ricLrSd^^:^ ?S4;r?h*i.Sct2fs3! is 

S?gnJiI= t\v?:d^1^?^i-T ?t":f 

^- Charaoteristlos Analyzed I n Relation to Achievement. 

'■haract»rUWcs"of °- ""'^^ t° detsmine ...-hat 

*f;l e^^ t-^e compensator, 

-.f black pupils in th.e =^fo;r-/„ t"°'V^,;- proportion 

:n black enrollment, -prSJonx^on ^of ' SpJS s^n^'^P^SJni ^tl^i-rf ^'^^ 

• n the school dur^n^ the IcA^ ^ir^^^^^^^-;:.. b-Gj...^:.os 

.;:hieve.ent level-in t'he Holi^ iT^^J^ ^^^e ^^^i^of 

•.??ance ^-S^f ^:?-\r'--d'' -^i-^^ ^'^^^ '^"^'y^^^ analysis of 

•uld a...o^;-.t n^;. fv : 'f-^'^^^ ^■■■■^^^^ characteristics 

Ui Jio ai .^ro. j; 'ni c!^--r ..-.'a papil perf cr::.ance . 



51 



ERIC 



Kost of the results frcrr. t::l2 anaJ.ycis v/ere inccnclusive ^ 
probably tecauc"^^ cT the overall lacr. of slrr.lf icant perfcrr:ance 
gain in the co v-jd data. The rsjressicn ar.alysis revealed 
no statistically .:ignificant relationship betv;een readir.:^ achieve- 
ment changes and tho characteristics of rr.obility cf p.,pils, 
school size^ grade size ^ degree cf econcrnic deprivation^ grade 
level or attendance rate. There v;as a definite^ but v:ea>.^ 
nonlinear relationship bex".;een changes in achievement betv:een 
1965-66 and 1966-67 and the proportion of black students in 
a school. 

The above pupil ^ school and prcgrarn characteristics 
have been suggested by rr.any as irr.pcrtant factors in achievement 
in the educational setting. It is important to examine here the 
logic from the TZy.PO study so that the student of research can 
note the reasoning behind the selection of these variables and 
the specific results cf the various analyses. 

For example^ educators and social science research per- 
sonnel have felt that there may be critical grade levels in the 
norm.al 12 years of public education during v;hich it is relatively 
easy to compensate for the effects cf educational deprivation 
and other grade levels v:hen it is relatively difficult. If 
educationally disadvantaged ciiildren are given special help at 
the appropriate ^rade level, they may be able to cverccm.e m.oct, 
if not all^ of their disadvantages. It has been generally 
accepted that the early grades are the m.ost ir.portant levels 
for such intervention.'' It is im^portant to note that in this 
study (one of the fev: to cover many grade levels) the results of 
several different analyses provided no evidence that changes 
in achievem.ent '..-ere related to grade level. 

The m.ain criterion for selection of schools to participate 
in Title I of the ^S>7A is a high concentration of children from 
lov/ incom.e fam.ilies in the specific school attendance area. 
Although the lav; specifies hcv; this Title I m.cney is to be allo- 
cated on the basis of economic guidelines ^ the standards used 
by local school districts to detesM^ine eligible schools vary 
greatly from district ot district. :^ach school in this study 
v;as classified as being high^ mediu.r. or lev: 'in tenns of economic 
deprivation relative to all Tit3e I schools in that district. 
Kone cf the analyses performed indicated any reliable relation- 
ships betv;e'en degree of economic deprivation and changes in 
aohie vem.ent . 

High student m.cbility v;as deemed likely to have an ad- 
verse effect on ccmpensatcry education for several reavScns. 
Firsts the range cf^ ccm.pensatory programs varies from school to 
school^ therefore a student v;hc transfers from one school to 
'another m.ay not be able to continue in the .vame type of ccm.pen- 
satory program. Secondly^ the disruption caused by transferring 
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.ay, have an unaeslrable Jf.ect =._«.e .tuaent's cverall^aoade.lo 

hi^b nobi it-. Vr,3S ir.ight have beer, ca-asea oy ir.ooil^ty ceing 
co?^eSater-:ith ether vSrlables v;hich also influence cnanges in 
achievement . 

Analvqes ^-r- utilized to detern-.ine the effects on achieve- 
ment on^'^Se'rrcpcr^Ion'c' blacK students in. a school and on the 
^v.c>^-e Pnron-^'rt of black students m a school. mere 
fop4'ed to b^r nonlinear relationship bef,:een chan.es^ in acnxeve- 
l^lt [>r.f.een ic6^-cc and 1966-67 and the proportion of olacK 
stfdenis in\ school. Of the 132 schools in the --pie tnose 
with rs-'at^vely lov; blac.i enrollments (less than <iO 0^1 cent; 
tenSed to sho-/statistically significant positive cnanges in 
a-hievcn-ent at the mean, the first quart ile and the ^ 11st aeciie. 
^h^JrouD of sanroie schools in the ^^0-59 Per cent blacK cate- 
lo?v sScSed sLtistically significant negative changes in achieve- 
^ i .s. ^^^^ ov,^ p-r -i-oo e±y<:f n^-.p.-rf.ijft. Since tr.ere was 
co"kd-.^oi-^''^'J-.'irtiorin'^ arr^ong the sample 

dislr?cls" it might be t.-.ought that t^-^se results simply^ 
difference amonc district.. Kov;ever, '^^^^fl^ff'-^^:t^f^^^^ 
■infiirflt--d t'r,at f-e relationshiD betv.-een cnanges 3n acnie/ement 
and pScentage of bLclc onrollment is similar within each of the 
districts . 

In the area of ^.lack enrollment, there were 26 of the 
sanol= schools •.;hi2h ch.-..ged five per cent or more cetv.'ecn 
!c65-66 and IC06-67. Within the 26 schools there v;ere 53^school- 
i^ado obs'=rv"ticns, hi of v/hich increased in percentage Ox 
S^ck s^ud^nt's and 12 of v.'hich decreased. Changes in achievement 
tiS scores at the mean and first decile for these school 
trades indicated that relatively large changes in percentage 
of black students in either direction are associatea v-ith less 
favorable changes in achievem.eno . 

The proportions of Spanish- speaking students in school 
districts -as also utilized to investigate relationships to 
achievement. Several cf the districts in tne sample nad re.a- 
Uv^iy large orcoorticns of Spanish speaking pupils out .ne 

o? ipaurih spe5;cini stud-.-nts a.^cng schools ■..■ithin eacn aistr.ot. 
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Attendance is assuir.ed tc be irr.portant in school achieve- 
ment and is also assurr.ed to vary v.'ith pupils* rr.ot ivaticn to learn. 
For the sarr.ple as a whole ^ there v;as a srr.all positive ccrrelation 
betv;een rate of attendance and change in achievement. In the 
area of attendance and its relationship to motivation tc learn^ 
increasing rates of attendance might be positively associated 
v;ith change in achievement: hov/ever^ correlations computed from 
the data shcv;ed no clear relationship bet/een these 'cv;0 variables. 
The available m.easures of change in attendance v/ere crude ^ since 
the definition of attendance differed am.ong the sample districts, 
and there v;ere no controls for other factors (such as vreather 
or illness) v/hich might have affected attendance rates. 

Recently there have been theories v;hich propose that 
the achievem.ent of students is related to the size of the school 
or the grade. Som.e educators feel that there is an cptimium 
size vmich Drom.otes better achievement • The statistical evidence 
in this study shov;ed no significant relationship oetv;een changes 
in achievement and the num.ber of pupils in a grade or the size 
of the school. 

Although the miain focus of this study v:as on changes in 
achievem.ent test scores and the relationship between those 
changes and selected pupil-school-environrcental variables, it 

from different levels of expenditure for ccmipensatory program^s 
on achievement. This v;as done by inclusion in regression analy- 
ses of a variable indicating average Title I expenditure per 
pupil in each sam.ple district. 

In order to relate expenditures to achievem.ent test 
results, a new variable, "effective Title I dollars per pupil, 
v;as created. For each sam.ple observation this variable was 
comiouted as the sum of average Title I dollars per pupil in 
1065-66 olus a ocrtion of the average Title I dollars per pupil 
in 1966-67. The portion used for I966-67 was basec on the fraction 
of the academ.ic year which had elapsed up to the date of the 
1966-67 test for each specific grade unit. This variable did 
not take into account any funds other than Title I which m.ight 
have been expended by the school district for com.pensatory 
program.s . . 

For the 11 districts having data available, there was 
a positive correlation between effective Title 1 dollars per 
pupil and chan-es in achievemer.t test scores. The correlation 
was statistically significant at the five per cent level for 
each of the four' m.easures of achievemient r change in the mean , ^ 
change in the first decile, and changes in the f1>"*^t ^1.^ Cnlih, 
quart iles . 
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At the level, of specific school aiHtricts, there also 
seei;-.ed to be a congruence betv;een cnang&s in achiey-crr.ent test 
scores and effective Title I doi:.?,.rs per pupil, 
districts v,ith the highest leve^ of Title I 
l£.r,se£t gams in achieverrient at the first 



Tne three 
funding had the 
decile . 



These .->'--'tive findings should be regarded as a very 
crude atterr.ot ieterir.ine the overall relation betv.'sen level 
of funding and c^nicverr.ont change. T.-.e findings -.i.-ere cased on 
h "'''1/ agsre-ated data ■•.•nich did not -"ive a true picture of ■.•;nat 
is 'hacoenin^'^at the grade level. Analyses of achievement change 
indicated that not only v.-as there great variation in perfcrr.ance 
air:on.:c districts, ' ... also great variation among schools ■.-.■itnin 
a district and among grades within the same scnocl. 

The Phase II study considered the relationship tetv;een 
changes in achievement and tne kinds and costs of com.pensatcry 
pro-ran-s at the grade level within san.ple schools in five dist- 
ricts. Tyiis more detailed examination increased the understana- 
ing of each of the five case studi-^s, but the small num.bers 
involved introduced too much sampling variation to allow generali- 
zrtions . 

A larse numroer of other m.ore minor questions and hypo- 
theses were also investigated using the data collected by ThMFO. 
V.TiU-^ t'-^e availab3e data did not orovide conclusive results in 
most of these areas, the referenced Tr.:.:?G r^.pcrts discuss the 
analyses and results in detai] . 

G. POSTSCRIPT FOR TH5 RESSA.RCHER 

Potential and practicing researchers in the social 
sciences will draw a variety of lessens from the actual experien- 
ces described in ■:his chanter. One thing is certain, Title I 
as the largest F-^deral Educational aid effort in tliis country 
deserves the best and the m.os-c scph.ist icated evaluation efforts 
possible. The TEMPO study just described was an early attem:pt 
in that direction. There continues to be a need to be wary of 
such m.eaningless statem:ents as are found in the National Advis- 
ory Council Report for 1^72. This report concludes with tne 
statem.ents: ''The Council's investigation in 1S68 in.dicatea tnat 
abou:- cn^-auarter of the programs t^er. were bringing results. _^ 
ToG^.-. the *6c-tim.ate is thac m.ore than one-third of the Title I ^^^^ 
Drojrvams are facilitating tne euucatiLa of disadvantagea cnilaren. 
To tne educational researches and policy m.aker, such statements 
quic'rcly bes: for specif ics--what Kind of results, positive or 
ne.^aoive, from where would the results com.e, (since there were ^ 
not studies vieldin.g hard data which could produce that type oi . 
definite conclusion). Is the goal of Title I to facilitate, 
:.e., to m.ake easy or less difficult? Is easy gooa or bad? 
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V;r.at evicier.ce or data v.-as used to determine facilitation of edu- 
cation for the disacvantageu? 

Th" pro'" passional educational researcher must den.and 
froro the federal soveri-ent , the state and the local educational 
agency the rignt to use a research design and tools v;nicn can 
cr'iv'^ Doli-y rra'-cers, ccnsurr.ers and citize-s specific ansv:ers 
and data about the orcsress of the disadvantaged child m tne 
iijT.'-rican Educational systerr.. The educational researcher n.ust 
r^irep.be-^- tr e ourpcce of Title I was to help the disadvantaged 
student, not to glorify teachers nor tc create jobs for policy- 
makers or adr.inistratcrs. Vrith th-is in mind the educational 
researcher rr.ust raise specific questions; he must use tnc best 
research tooDs available for his task; he must never be satisxxed 
with a subiective resoonse to a o.uestion such as Is ocnnny 
reading better after his 10 months in the Title I rem.edial 
readin-? program?" He must be able to determine this on tne 
basis of vel^-able and valid measures '.vhich have been accurately 
administered. He m.ust be able to defend the representativeness 
of his sa-mple, the cost-benefit ,-rocedures used, etc. r.e m.ust 
do this in order that present programs and procedures can be 
improved so that the future Johnny's who are educationally 
disadvantaged can be v.-orked ".rith on the basis of enligntenjr.ent 
instead or "m.ore of the sa:ne" or random experimentation. 

AjlI Ui OUX' pOJ-J-C^ iiiarvxii^ c>.^.._. _i . s^x s^j- — ^ ^ w 

ov^^e this tc the consvar.ers oi' educations-students. 
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%'his study v;as funded by r;ontract3 }iEV;-05-67-->:, and OIC- 
0-8-c2o46:;-3':i3(010) fror. LHE;;' and 02 respectively, "..everthe- 
less, the conclusions of this arxicle are tnose of the authors 
and rr.ay not necessarily lepresent rhose of TZy.PO, Lh'i-V.' . or OE, 

Sv.'hile Dr. Kenneth Gor.' ■ v;as a r.ember of the TZMPO team 
during the analytical, roper rlting. and case study phases 
of the projects, several cz':. r people v.-ere involved fcr^^even 
longer periccs.' All prelect staff contributed to the T--.;.:?0 
reocrtc' describing the project results, parts of v.-hich hd.vo 
been used i:. this surj-.ary article. Lr. Harold Asher, varager 
of tlie Washington office of TUXPC , v.as responsible for the ori- 
ginal concootualizaticn cf the study and provided genersl 
manager.ent suoervision throughout. Dr. Joseph VJ. Harrison was 
responsible for the early design ana data collection phases and 
Dr. Ernest Mosbaeh v;as responsible for the later phases of data 
analysis and ■-■ecort writing. M>-. Ronald Frola, Mr. D. Stephen 
Kayer, and Dr. Charles '.v. indie v;ere heavily Involved in field 
data collection ar.d data analysis activities. Mr. Mark Adler 
coordinated t/.e processing cf the extensive student achievement 
test data. TEMPO v;as also greatly aided during^ this study by 
Dz- . Alice. Kivlii; a..u Dr. Jcsep"; '..'i'.olcy cf the DH£';.; Prcgrar. 
Analysis Group, ;.-ho successively exercised technical monitoring 
responsibility over the TEMPO contracts. 

3surve y and Analy ses of F.es ults Pron^. Title I Fun ding for 
Co niDensa torv " - vu c;.t Ion. r (ca..ta raroara, eaiiforr ia; ihv'.r^, 
Oei:eral n.lectric .oir.paiiy Report C7IMP-II5, lyoO. 

^■Analysis of Compe nsator '- Educaticn i n Five Sch ool D istricts 
(2 Yolumtsli ToaZTaT'Jar c ara , al iTcr n la i;^MrO. General electric 
Company Report ceTMP-93j IS'^--)- 

^United States Supreme Court decision, Brov.'n vs. Po ard_of 
Education of Tooeka. Kansas, 19>-. This decision overturnea tne 
Plessy vs. r'ergaso-/ ruiirT^that racial segregation was permissible 
if schools were separate but equal. The 195^ decisioi: mdicatea 
tnau separ-ate but equal has no place in the U.S. system, 

^Bailey. S. K., and J.'osher . Edith K. , ESEA- The O ffice of 
Education Ad-.ini -ters a Law : (Syracuse . h'ew \'ork; Syracxise vn.iver- 
sTtv Press. l,-v">. p. i. -cr tne scj-'al scienc>= researcher who 
desires to'kn.o.: '-.ere about the ferc^^s at work at the ti:.-.e of 
the passas-e of ESEA, the authors of this text offer thorough 
discussions of the politics], social and religious influences 
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coupled vn'th t'r.e re-alitic-is of por^^eivoa irr.portarrje (ry 

Cc^recc nvA t:.e .:./:ocut : /o Arcaj c:' the Cfj^i:;e or Ziujation, 
pp. 

TThe Smith ;-:U'7;he3 Vocaticna.'! Act of IS'? v;as evid^r.ce of 
one of the fir^^t entries ir.to e'iu :ia-j : cr.al f ir.ancir.-x for the 
federal ^toverrir '^rr.t . It created a ^^artnerc/r.ip betv.-een the locals 
state and fe>cra2 r;cver!.:r.C'r:ts for t^ e purpose of preparir.?; 
students for crrpi oyrr.ent , It \:a.z ncz until the Vocational Air.end- 
ments of lSn3 (fclio-.;ed by the A: .endirer.rs of 19^9) that the 
Federal Governr.eni significantly restructured their role in 
vocatio.val education.^ The ber;i'nn:nv; cf Federal Govsrnr.ent 
fir.ancinp: of educational progra:-.c in IS'17 v:as follov;od by a 
nearly total al.*sence of additional prograrr: funding until the 
post V.'orld /Jar II period. Federal e/.pendioures have gone from 
7 icillion in 1937 to the 30G rr.illicns* at tne present tlir.e for 
vocational education. 

o 

In response to post-Sputnix concern about our scientific 
and research r.a:.po'.:er capabilities Congress enacced the :!ational 
Defense Education Act of 1^55 providing federal financial sup- 
port to increase the quantity 3nd Gi;ality of our engineers, 
scientists and foreign language specialists. This financial 
support n-as for both secondary sch.ool and higher education 

^l"'he Cooperative Research Act v.as passed in 195^* This 
Act had its basis in the preniise tnat the scient if ic ^rr.ethod 
applied to education would equal iir.proverrient . in 19^5 this 
entry by the Federal Governr.ent irito the educational research 
field v:as expanded into Title IV, ESFA resulting in the funding, 
of 12 regional educational laboratories (v;hich generally eir:phasized 
developrr.ental research) and 3 university-based R&D centers 
(enipnasizing basic research). 

-'■^The Eleir.entary and Seco!:dary Education Act of 19^5 was 
coir.posed of five ir.ain ':ritles . Appro:-: ir.ately 33>^ of the total 
funds authorised for ESEA v/ent into Title I, Better Schooling 
for Educationally Deprived Children. The rerr.ainder of the 
Titles vjhile not specifically concerned with the disadvantaged ^ 
would theoretically brin^ improver.ents to the educational system 
and, in turn, benefit Lhose children v:ho suffer educational 
liandinaps. Title II pro /idea grants for the acquisit io)^. of school 
library resources. textbocKs. and other printed and p;iVlished 
r.aterials; Tlt^e .II pro/iced fu: d::..r on a cc:::pet it i ve casis for 
supplcr.en.tary edvcational services and centers in order to stirr.u- 
late exen.plary progran.s .;n.icn. could serve as rriOdels^ Title IV 
(noted above)'^and Title V which provided funding to stre::gthen 
state departments of education. 
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l^ Hl-torv of T itle I. L3?:a , (pf rtn.ent. of Health • education 
and V;elf are . office 01 "lau ja-clc:. ^ ll;7^/ p. 3- 

^%or example, in FY l^c3 36. 1;' cf expenditures ;;ere for 
reading projects^ S-C/l for liorary, -^.1:1 for counseling and 
guidance^ and 3.7;^ for rr.ather.atics instruction. 



13 

tunit 



-^Coleman , Jar.es, et al ^, gquality of Zduc, at ic:.al Cppor - 
y,(U.S. Office of Euuca"^ Ion /OrO , July Ib^--;)- 



l^Section 402 of the Civil Rights Act of 196^ stated The 
Conr.Lssior.er of Education shall conduct a survey and :::a>e a 
report to the President and the Ccr.rre3s, v/ithin tv:o years of 
the enactir.ont of the title, ccncernin.:: the lac>v of a vailac: lity 
cf equal educational opportunities for individuals by reason 
of race, color, religion cr national origin In public educational 
institutions at all levels in the United^ States , its territories 
and possessions, an.d the District of Columbia. 

^^op. cit. 

■^^The nearing deadlines for scir.e cf these review cycles 
were the reason for the original tight schedule for the present 
study, which later had to be stretched out somev;hat. 

17 

Th.is v;as the same organization that later v:as upgraded 
to Assistant Secretary for Program Ana.iysis and even later 
became the current Assistant Secretary for Planning and Evalua- 
tion (ASPS). 

■^^Hemem.ber , this v;as during the first tv;o years of a v;hole 
nev; federal approach to helping,, local districts with their compen 
sator;, educaticn needs. It did become ::iOre com.mon to enccun.ter 
concern and possible hostility from prograin staff that felt 
threat enea by outside evaluations during the evaluations cf speci 
fic iede>-al progran.s that cam.e several years after this project. 
This v;as particularly tr- j .-.'hen either the existence or technical 
approach of federal assistance programs v;as very controversial 
or' under pressure, either in the political or educational sectors 
It v:as not until after the TEi-TO evalu?,tion (perhaps influenced 
by its lack of positiv results) that Title I becam.e heavily 
criticized. 

IS^'See DO. TI ^.5 of tliis chapter for explanation if "Fixed 
Graae'' and Longitudinal'' a? vcilic.-d i.: this specific study. 

20sir^.ce the dem.and for evaluation across programs an.d 
across school districts has ii-tensifled in recent years, the 
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Office of Educaticr. has spcnscro:: zY.o development of criterion 
referenced ter.ts ll.-:e tr.o ]'.e.tic:\nz Az2ezzr:.ei\t of Scr;cat ior.al 
Progress. This prcgrsr. is beir.p; carried out by the l^ducation 
Corrir.ission of the States under contract to the U.S. Office of 
Education. The testing in this project .'.'as begun in 19-9 ^"^-^ 
repcrto are ncv; available on science^ citizenship, reading, 
literature and social studies. Contact Dr. J. Stanley Ahir.an, 
Director, IccO Lincoln Street, Denver, Colorado 60203 for 
further inf orrr.a^ion . The Office of Education is also supporting 
the ::ational Ancror Test Project aired at improved cross norm- 
ing of similar achievement tests *./hich v;ould enable the educa- 
tional reseacher to compare a variety of achievement test 
results. For further information about this project being 
conducted by Educational Testing Service. Princeton, i^ev; Jersey^ 
contact project m.onitor Charles Hairj,,er, Bureau of Educational 
Statistics^ Office of Education, v:ashington. D. C. No date is 
currently set .Cor ccm.pletion of the project and no publications 
are available at this tim.e. Alternatively, several exploratory 
research or program evaluation projects h.ave designed and 
administered special performance tests, such as for the eauality"" 
of education study (Coler.an Study) and several longitudinal 
studies currently being conducted. In each case^ funds r,any 
times over those comamitted to this TEMPO study v;ere needed. 

^^California State Department of Education, Ev aluation 
of Title I Pro jects of California School s . Annual ' Report lScT-68^ 
"SacTIImento^ 19c o, 

^^California State Departm.ent of Education^ Evalu ation of 
ESEA Title I Projects of Calif crnia Schools , Annual r.eport (O-Yl, 
Sac r a,mie n to ^ 1 97 ^ • 

^^Edjjt.^i^ior. of the Disadvanta.red : An Eva luation Report on 
Title I, ^le : ■ontar7''ana c ezc-ucry ^au9?Ti c v. Act o:' 1-;::;; E'C,^ 
19t:c . .:asningtcnr''''('J.sr i^epartm.ant o: r.enitn^ j^aucaticn ana 
TTeTfaVe:, Office of Education; U.S. Governm.ent Printing Office^ 
April 1970). 

nil 

Students of educational research m.ay^ v;ant to review this 
study for the socio-economiic inform.ation .';hich s sum.marized for 
program, participants and for the discussion of teacher assessment 
of the disadvantaged child and the relation of achievement to 
this assessment. 

^^Gordon. K. P: Gordon. B.: and Frola . R.; Evalu ation of 
Title I 2SEA Jom^ en^ator y Edu.'aticn . March 3 971, lTx7723 , Jeneral 
Electric ^^.-.r^., .■.acningtOi:^ J. C . ^ 'p . 

26ibid^ p. 3. 
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^^^ Iblci . , p. Y. 
^^Ibid. , p. 5. 

29Adciit icr.al ir.f orrr:at ior: Ci these metiiods can be obtained 
in rr.ost standard testbcoKs cn statistics. 

30sor.e other eval-^.atior studies by US02 have included the 
adrr:inlst>ration of special achleveir.snt tests, ncteably the 1563 
Colcrr. n Study (Equality of Zdvc^ticnal Cpccrtunity ) , and the 
IS'7? study of the effectiveness of Ccn.pensatcry Education. Each 
v:as fun.ded at over a laillion cellars. 

^■^The follov;ing definitions help clarify necessary techni- 
cal terms: 

Grade s:-ore, Cirade Equivalent or QE - Scores reflecting 
the grade claccr.ent of pupils for v:hcrr: 
the given score is the average of norn. 

Percentile score - A score or rank indicating the per- 
centage of pupils the standrrrdi cation 
group^at tne given grade placement having 
scores less than the given score. 

Rav; score - The number of correct answers in a test. 

Standard T-score - A score representing the placer^.enfc of 
the individual in a norm population of 
norr:ally distributed scores around a icean 
of 30 and standard deviation of 10. 

32vaticnal Advisory Council cn the Education of Disadvan- 
taged Children: Anivial Eeoor t to ti .e Pros ident and the Vc-n -^ress 
1 97 2 - Educatin-^ tr:c ^2 sad /anta.r c a : IcTT' . : : ere ";.e .'rr'i: euT ] a s n - 
ington^ L^.C.; D:::^.-. i:\ib. I . o . ['^1 ) T^:-Vr. 1 ^ T^^IT! 
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Eailoy, Stephen K. and V:o3hor, Edit:'. :<> ZSZA-The Office o f 
Ed ucation Ad rr .in: sters a Lav;, Syracuse^, liev; Yorr:: Syracuse 
Universioy rre.-^s^ l^rcc. 

Eerke. J.S.^3ailey^ Stephen^ et al , Federal Aid to P>ablic 
Education; :.nc henefits ? 'Syracuse^ ll/i. ; Syracuse University 
Researcn Corporation , 1^71 • 

California State Departir.ent of Education^ Evaluation of ^ ESZA ^ 
Titl e I Pro-^'ects of California Scnool s . Sacrarr.en-co . «CaT:.iornia : 
State Education Lepart-eno Co-- 0, i:^^-3CQ 11-68, I968. 

Connecticut State Eepartr.ent of Education, Connecticut Ev aD.uation 
of Title 1 of the Elen-.entary and Secondary l^ucat ion X ct^ r 
FiscaJ Year 19:^9 * HaFcTora/ ^.onnecticut ; L-l\^lsion of Ins o r uc t i c na 1 
Services^ lS-9- 

Connecticut State Departr.ent of Education^ H ope for Con necticut ^ s 
Dis advantaged Children ,- Hartford^ Conn • : Bureau of ms-crjc-cionai 
Services --ulleoin 1.0. 98^ I966. 

Connecticut State Departrient of Education and the School of 
Education. University of Connecticut^ Achieveir.ent and E olated 
School Fa ctors in the Largest Cities oT Connecxicuo > r.al-oTord^^ 
Conn. ; r.ureau 01' rrogram ijev^lopir.ent ^ April l^jf'O. 

Council of the Great City Schools^ _ Title I in the Great Cj.ty 
Schools- An An a 1 -si s of Evaluation ?ractices_ ancT Exen:pla ry 
Pro i'ecTrs". ..asnTngoOn^ i.u.: r.esearcn council of tne Jreai; City 
Schools 7 July 1970. 

Frola^ R. and Harrison^ J.^, et al. A Survey and Prel iminary; 
Cost-Eenefit Ar.alysis in Sler,entary-Secordary Educatio n ( Interim 
ProgressEapcr u ) a sn 1 ngton^ u.C: Jeneral n^lectric Y-.-. Jr ^ 67^ 
TMP^89, 1907 

General Electric TEVPC^ Survey and Analyses of Results from 
Title I Eun dinr for Co:- pen3au c ry ^z^aucaiion . Santa :,^arcara^ 
CaTii"^^ni.a : Jenerai ^lecuric, c? ivp-llp^ l^-oS. 

Glass. ::ene V., et al. Eduvation of the Eisadvant a;:3d. An 
Evalu ative Eeoor T^cTT^ Title j.l e:-vnTrry ana St:c c: Kiary \.\\2at±on 
Act cT'l-v EouTaor. Jciorrdo; ^.niversioy of uoToraao^ O^o- 
oratory of Educational Kesearcli^ 197C. 
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